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Troubleshooting of complex equipnSents in the military remains a problem 
for many reasons— som^ technical . some 'administrative. At the working leveT, ^ 
. a major key to succesi.is the technician's Information base or "cognitive map**^ 
of the relations that hold among present test symptoms, normal indi cations, 
and the set of pos^^ble. defective units. If this Information base Is reason- 
ably complete and correct, then" the technician can usually converge on the 
trouble In a reasonable time. Many equipments a^efso complex^tHat an ordinary 
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technician canRot be^^tted to master the informatton base\ after only a few 
months of trai.mng-^ancTexperiencei . - y ^ 

. J?{^l'?lQ^'''^^L ^PProaches to the troubleshooting problem attempt to. improve 
the selection,,riK)t1vation. and training of technicians. Among the aspetts^wWK 

considered are methods for enhancing the understanding of pKyaEl 
Tni hierarchical analysis and practice of trSuMfeot- 

ing ^ub-skills, and the general logic of searching behavior. The 
approach tries to "unburden" the technician by providing most of th 
tion base in special booklets, diagrams, computer programs, and dire 
fK^wK^* 'jaintainabililty approach concentrates on designing e 
^?rL-?K? will b^ easy to troubleshoo.t-for examp^le, key test points ar- 
accessible, aod sub-umts are arranaed so a? to fariiifafo Hi;»nn«c<c 



^ , .... ^-""its are arranged so as to facilitate diagnosis a^d^ 
replacement. When psychological, aiding, and maintainability technologies are 
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Itti Till : the troubleshooting problem can be "solved", within the'present 
state of the art, for the great majority of equioment iteuis. ' ^ - ^ 

hor troublesome equipments which ar.e already in service, procedural ized 

and'l^nfn''^^ ^'^^^^^^^^ ^"^ probably the best Mate so?uJion, ' 
and many one-term military technicians might well devote most pf their service 

iosT^H^'^'r.?^ ^^ds ."'"St be specially preparedriested! and 

r^S?L'"l'^'j: T.'.^^ ^"'^^ned from a behavioral standpoint It Sou d Se 
Sis ;n??h^J%J^'^ ^^^ T* .effective aids.will not find all 
.inHprl^^nJ"« flJ"^^ services will always need people with relatively deep 
understanding of the systems they have -to maintain. To provide such peoole 

nh!^r^^JLT^^l°^'"'^'^°P^" visualisation and memory of loZ\ex 

cpf n I^h • ^^^^"IPg o^ 1009 branching sequences^ individual dffferen- 

i!ed devices ^^^^^ e">^^"ce human reasoning by small computer- 



at 



/ 




\er|c 



4 
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SUMT^ARY 

Troubleshooting of conplex equipments in the military ranains a 
• problem for nwiy reasons— sc»ne technical, some administrative. At 

•I , 

tihe worlOng level, ^a major key to success is the technician's 
information .base or -c^nitive map" of the relations that hold among, 
•present test syiDptoms, normal indications, and ttie set of possible • 
defective units, if this information base is reasonably conplete and 
. correct, then the technician can usually converge on the trouble" in a 
reasonable time. Many equipments are so complfex that, an ordinary ' 
technician cannot be ejpected to roaster the information base, after 
only ^ few months of ttaijiing and experience. 

Psypholoq^cal approaches to the troubleshooting problfen attonpt 
to improve the- selection, motivation, and training of tfct^lidtans.. 
Among tha aspects »whicl;i have been" considered are methods for 
enhancing the understanding of physical relations dn-equipment, thef 
hierarchical analysis ajK3 practice of troubleshooting sub-skills, and 
the general logic of searching 'behavior. The aiding ^proach tries 
•to "un^rden" the technician by providing most of the information 
base in special booklets, diagrams, c6np±er programs, and directed 
test sequences. The maintainahn i^y approach concent fates on 
designing equipments so that 1±ey will be easy to troubleshoot— for 
example, key test points are made accessible, and Sub-units are - 
arranged so as to facilitate diagnosis and replacatien\; When 
psychological, aiding, and mjintainability technologies are fully 
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applied, the troubleshooting problem can be "solved," within the • 
. present state of the art, for the great majority of equipment itons. 
For troublesome equipments which are already in service, 
procediralized aiding of trdubleshooting. activities is pc^phly the- 
best imnediate solution, and many one-terrp military techftioian? might 
well devote most of their service to application of these aids". .The 
aids must be specially pr^red, tested, and debugged,' and they must 
be designed from a behavioral standpoint, it should be rec6gnized, 
though, that even the most ef fective >ids wi*l not find- all troubles, 
and that the services will always need people with relatively d^ 
understanding of the systems they have to maintain. TO.prbyide such 
people, research is needed on such topics as the visualijzation and ' 
memory of conplex physical events, the Reaming of long branching 
sequences, individual differ^ences in search ■:^havior, and ways to 
enhance human reasoning by small computer iaejS devices. 
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Troubleshooting. Complex EJquipmeftt in the Military Services: 

Research and Prospects 



SECTION I. INnmJCtlOJ 



Oie of ^ the first- experiments on human troubleshooting of 

electronic ^^LTjpnent*^^ conducted in the summer of 1953 at the Long 

Beach N«val Shipyard. Electronics technicians from Navy ships 

searched for faults inserted into radio ahd radar circuits. The 

subjects also, had to troublesho6t a "simulator" representation of the 

sam? circuits. Th^ main idea, wa^ to see if fault-locating behavior 

on the simulator correlated with performance on the real equipnent; 

in fact, it did (Grings, et al. 1953). 

•fliat early worK was sponsored by the Office of Naval Research 

because o^E the dramatic increase in equipment oompl^exity that • 

:curred after World War II. The new radars, fire control devices, 

missiles, analog computers, and ooirenunication sets were vastly more * 

corplex, and often more unreliable, than%the ^ipnent of the 1940 's, 

and all the military services found themselves with a maintienance 
t 

headache. While the increased capabilities vrtiich the ^dvanced ' • 
equipment afforded were highly desired, much of the tine /Gie fancy 
new gear was down, and often there was nobody around who knew how. to 
fix it. As early as 1953, Carhart's Rand r^rt was oalling the Air' 
Force maintenance ^situation "intolerable" (Carhart, 1953) , and one 
could bear even stroi^r adjectives f rcro qperational military people 
who had to us^ the new technology, and^had to live with it v*ien it 
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failed to cerate. 

More than a quarter-century later, the troubleshcSbting problem is 

still with us. Ships, aircraft, missiles, and control centers are 

-\ . . - 

still plagued with unf indable troubles and marginally performing 

> 

equipment. Many expensive itons never perform up to design \ 

expectatioris, or are unavailable at crucial manents. Maintenance 

still requires a lot of people and a lot*of money, with nearly 60 

percent of total life-cycle equipment costs attributable to 

personnel expenses (Blanchard, 1979; Shr^ver, 1975). Maintenance 

difficulties are not confined to sophisticated electronic; systems, 

either. Consider this astouraSing quote from Secretary of 

Defense Schlesinger, after the Mayaguez incident of 1975: 

"...The thirty^one year old, carrier Hancock... operating 
without one of its four shafts... never reached the scene. 
Ttie h^icopter carrier €)kinawa, . . .witlt part of its boiler 
plant off the line... also never arrived at the scene. The * » 
escort vessel Holt, the first ship at the scene, had power- 
supply problems, and consequently its main battery^ was down 
jthe night before the engagement." . . 

/ . ■ ' ' . ' ■ 

All of these difficulties occurred in a combat area following a 

major war, "when the. Navy had days of warning. Evidently, things are 

Still "intolerable" ^ military maintenance. . ^ 

This report was undertaken in the Belief that now . is a good time 

to take stock of the troubleshooting, behavior prcilem in the 

military. The whole field is changing and particularly so in the 

electronics dcmain. For one thing, we are in an^era of sharply 

improved hardware reliability. Aftep many years of prGmises and 

disappointments, hardk/are is fincdly mi^^ng its claims, lliese days, 

a hew mini-conqputer with dozens of terminals may run for days or 



* Page 



weeks without a main frr^'-z fa/ilure. This is true "ultra- ' 
reliability.- Already^^>itegrated-circuit components are manufactured 
to be "fault-toleifant," so that if something fails the system keeps 
running as before. In a certain sense, the failure never .happened 
because the user never knows it. But as Rouse (1978) points out, 
relatively rare breakdowns mean that a maintenance technician will 
not get much 'practice in troubleshooting. Another big change is the 
micro-circuit technology, which makes it feasible to localize a fault 
only down to a board, or chip unit. At the same time, cheaper 
and larger computer memories make it possible for systen designers to 
«np],oy very long operating and fault-locating programs in a small 
unit, and this mea^th^t a technician often has to ^be -something of a 
' programmer. So we can see that technicians will have to become more 
oriented to logic and software and will be less frequently concerned 
with the old soldering, wiring, and parts- replaconent skills. We are 
probably at a crossroads, in the aiding and training of 
trbubleshKJoters. Handy and comprehensive "briefcase" troubleshootiog 
guides are already here (Rigney & Towne, 1977) . A portable test unit 
as big as a TV set can store a whole book of systan information on a' ^ 
single floppy disk (De Paul, 1979), and all of the data can be 
interrogated inst^tly. Not loii^ from now, such a oonpapt ^ide will 
tell the military technician vftiich check to make next, will evaluate 
the results of all ch«:ks made, and will furnish a'hard-cciy printout 
<tf every troubleshooting st^. And these aiding functions may, on 
occasion,, be instantly transmitted from Crandte ship or field site 

♦ . * . * 

to an analysis center, for further analysis and advice. Oliis l^nd of ^ 
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— -detailed aidijig can be so effective that much training might well ^ 
directed at teaching the ^id, rather than teaching the usual circuit, 
h^aulic, or electro-mechanical theory. There will be arguments over 
su6h aiding and training, but it is clearly on tiie military horizon 

* ' • 

for the 1980 's.' Another prediction is easy to make; more systems 

will be digital. As one writer asks, "Is everything going digital? 

Whatever happened to 6SN7 tubes, low-distortion amplifiers, and 

feedback circuits?" (Gasperini*, 1976) . 

• If indeed we are already in an era of ultra-reliable but more 

conplex digital equipment, in which technicians will be aided in 

ir work by extremely "smart" test sets, .then there are many 

interesting questions. What can we expect of our troubleshooters? 

Exactly what will their intellectual require?ients and difficulties 

be? ^How much efficiency can we anticipate? What administrative and 

planning considerations, should be most crucial regarding the miliCcury 

technician force of the next couple of decades? How should we train 

and "transition" all of the people required? What contributions can 

be made by disciplines such as the psychology o£*reasoning? ' 

Hiis r^rt was written primarily for the military research 

comnunity, which is a network of technical conmandSf laboratories, 
/ 

0 

contracting bureaus, industrial contractors, and academic institutes. 
As a secondary gwdience, there are numerous training, support, and 
operational personnel who run the technical schools, who have. to face 

t ■ r 

troublestooting problems in the field, or who prqpare technical 
manuals and autcmatic t^st equipment. Such people arf oft^ engaged 
in the delivery of a specific systgii, course, document or procedure. 

■ ' ■ • • *• • 
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Wfe also hope that some miaitary nanpower planners will consider scsne 
'of the issues rais^ here„ Ihere ar-e rather,' few manpower geople and 

' - " ♦ » 

they are^not particularly visible, , but they are the pnes who decide 
•how many Data ^^bns Techs we wHl have, in 1986, and. thef r decisions 
can affec^ Idng^range. mij-itary prospects. Ther^*is no n§ed tq plian 
for an expensive new E)CK installation on a ship if the prime • 
equipments will not be available for 'use. With so many big changes- 

•Jin the hardware,- software, and aiding .capabilities^ manpower 
projections wilj be more difficult and more critical, and they 'shoul'd 

t>e based on what we know about th^.,)5ey. betiaviots* that will be needed. 

, Sec^^icffi II Examines ^the logic of diagnosis, and also reviews 
material fr^ academic psychology regariain^ human-search'performance. 
Ohe next chapter takes up aiding and training issues, many of vhicH 
are being pursued right" nou' by DOT> agencies, labotatOries, and 
contractors^ Section IV looks at the maintenance tefchnician's work 
. from the "job, design" standpoint, and /explores his job" satisfaction ' 
and motivation. Hhe f inal sfection lists some research 
reccmnendations, including nineteen specific projects that might be 
considei;ed by agencies like ONR, AFX)SR, and ARI.' There are also a 
few administrative reconinendations. 

The main qonclusion of this r^rt thesis dan be quickly 
inparted.- Troubleshooting of veryjOanple:i| systems is difficult for 
numerous reasojis, t?ut the critical factor is that the technician's ^ 
opgnitive ro^ of essential physical relations (electronic, hydraulic, 
electro-mechanical, and so on) in a ccanplex equipment is often 
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. incGRpIete, vague,- or incorrept. A3 long as this is- so, any series 
of checks and test readings, though, apparently well motivated and . 
aocofljpli^ed, cannot "close in** logically on a faulty unit. All the 
,Vtechrii<jian <^ do in this environiftent is to keep making checks in the 
hope of. finding some drastic test indication, or to engage in ' 
probabilistic or mass r placement of ai^units. The conventional . 

approach has been to provide general "theory" training in electronics 

* • 

y • or hydraulics for the technician, Arm^ with this theory, and with 
the assist€ffice of technical manuals, .the techn4.cian supposedly coald 
generate his own fault-location sequences. But modem prime 

'equipne|ts are so oonplicated that it is not reasonable to expect an 
ordinary military technician to know thpn well enough^to originate 
effective test sequences for himself, after *a few months 'of training. 
It is, then, not so; much the technician's ri&asoninq that -is at fault, 
as it his information base regar-ding' the meaning of the checks and 
test readings he makes. Given a sharp map or model of the 
elimination logic that prevail J between observable \te§t sfymptans and 

" the set of possible troubles, a technician can usually solve a 
troubleshooting problem within a reasonable time, even though his 

" logical path through the netw)rk is not the optimal or most efficient 
sequence. If the equipment is es^tremely large or oomplex, then some 
kind of procedural ized or step-by-step insructions will be 

- « 

necee^ry. s- 

It follows that much of the training and aiding effort shoiiLd be 
directed to pr^ring a special symptoA-malfunction model of tt|l^ prime 
eqCiipinent, and making that model accessible to, and usable an 
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ordinary trained person. If there is sufficifinf hnn i^tpI , 
dgtefmination to achieve'^ this, as in the oonan fi rcial conputgr 
industry. ^th^n the major technolOQiCal probla n s of aiding can bp 
solved, at le^st for the ma-^oritv of comp lex systaT^s -. • Right nov, arid 
for the next few years, the best , solution probably. ife to provide 
special "aiding'" or^doclWntation to the technicians, to Have 
this documentation so effective th^t it will actually guide 
troubleshooting down to the ^ropr^ate oonponent depth, and to give 
'the technicians intensive practice with these materials*. The 
COTipeteht fir^t^edn technician thus becomes an extremely fluent user 
of conplicated booklets, charts, ^ software routines. He will not 
often originate test sequences himself. .Rather,- he will follow a . 
> definite "tree logic" or other algorithm that will guarantee 
localization of most failures; If the documentation and "associated 
procedures cannot locate certain troubles,, then the aiding peckage 
, must be imnediately modpLfied so that it wiU ^accomplish that task *in 
the future. In this scenario, w4 do'.not l^se into "the conception of 
"train^d-ajpe maintenance, " where an untrained person blindly follows a 
conqplex test, routine, and does what it tells him to dto. Our 
^chnician will still be a highly skilled person. It will take great 

care' and judgment on his^rt to carry out the a^fopriate 

♦ ' ' ■ 

procedures* The aiding technology to support the troubleshooter is 
already here, and has been proven in about a dozen demonstration 
projects (Shiver, 1975; VanHeroel, 1979; Foley, 1978) . lliere will 
< airways be a need for military maintenance people who can go beyond ■ 
the procedural ized aids; and cai) solve the difficult problema,. These 
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people will have tq possess a deeper understanding of the systems 

they work on; perha^ a good )bodel here would Ipe the "flying squad" 

specialists^ in the^oomputer industry, who are extremely carpetent in 

both. hardware and software aspects. of big oonpjters. Training such 
' y , 

people, asd p£;oviding materials fot them, will be a challenge to the 

' • . . • . .. • • 

military' Pier ^onni^l conmiflnity. . ' 

Efficient teroubleshooting £aD be achieved in the military. There 
are many suc^ssful^xamples from Navy sutmarines, from the special 
weapons ar^/ and from certain aircraft ^sterns. Also, many good 
people VN the^ military system th^ that they are helping the 
troublesshooper. Examples of these are the people who originate, fault- 
location ^hemes, the school instructors, and the manual writers. 
Accordim to our view, many of their efforts are far less effective . 

thah tfley could be, and this is because ttiey do not stay jclose to the 

If",' ■ / ' 

actual cognitive situation faced by the military technician. Onjr 

,of this report, then, is to explicate some of the' ways that 

roveQionts can be made. > ' . 

Sibliwraphigal Hots* ^ ^ • 

Uie troubleshooting literature is ^widely scattered in about half 
a dozen major clusters. For behaviorally oriented studies. Rouse's 
privately-circulatea bibliography is the best. (Rouse, 1979). In . 
fact. Rouse's list of references in one of his articles constitutes a 
reasonably good library on the subject (Rouse, 1978)% Johnson's 
(1972) text is the standard reference on the psychology, of 
reasoning. F&ult-tree methodology, probabilistic troubleshooting, 
and the associated mathematics are sunndi^ized in the ONR-sponsored ^ 
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repoi^t on reliability and fault-tr^e, analysis (BSfl^, et al, 1975), 

and in the Ross (1970) textbook. Far maintaiiiability program 

• " ■ • */ • ^ 

management, the best general ^urce is still the Blan'chard and Lowery 

(1969) book; although the microcircuitry revolution has dated some 

■ ^ of it, this text* h^E thorough treatsnents of maintainability 

prediction, tradeoffs, and demonstration procedures. Thdre are 

several new books oh the logic of digital troubleshooting (Gasperini, 

1979; Coff ron, 1979) . The computer-science approach to complex 

search problems is treated by Raphael (1976) J There are many guides 

which will .tell how to design a maintainable device, a good example 

being the one by Morgan et al., (1963) ; State-of-the-art in military 

Ihaintenance aiding and training is weU reviewed in the MTBC-Orlando 

conference proceedings (NTBC, 1975; OTEC, 1979), and in Foley's ^ 

•sumjary report , to the Air . Force (Foley, 1978). Foley's surmary is 

especially inportant, because it lists every experimental evaluation/ 

of maintenance aiding to- 1977, and ity^so provides strong ^ ^ 

adninistrative advice for impl^ienting^fully^proceduralized aiding^ 



systems. 



SECnCW II. LOCATING A PAIUSD (XMPONENT 

The XjOfljc g£ PiftcgWSiS . 
t '■ ■ Much troubleshooting is very simple. When scinething goes wrong, 
or most o£ the possible causes of the fault are listed, and'' are 
^thsn successively elimiJiated. l!lie process is continued until only 
^ one culprit renains. Diat itan is refairSi or traced and the 



17 



Pa^e 10 



probleh is solved. Or maybe yOne component is known to be likely to 
fail» it is Uierefore sunnarily r^laced, without teh of ar| 
atteraptl^t fault-^tracing. Only if this "obvious*' replacement fails 
to clean the trouble is any further search undertaken.. Such 

\\ "■■ ■ 

troublesmooting is applied in all practical tzi^des and professions, 

\ - '4 

I * 
1 \ ' 

and in daaly life. It seems so natural that it may not ^en be 

\ ■* • • . » 

formulated in an explicit way^ Technical difficulties arise, of 
course, w!^ the list of possible causes is incomplete,, when it is 

• * * * 

iiipossiblel to eliminate .sane possible causes, or when^there are ^ 
great many related "partial causes." But the ooninon-sense conception . 
of trouble-isolation sta^s up very well, and is especially effective 

/ 1 . ■ , 

in easily visualized physical systems, 'such as a plumbing layout; • 
' When sy'stems get more complex and the number of alternativjes 
^ becomes larg4, a sinqple enumerate-aifid-elimihate. strategy will \ot be 
practical, it was just barely practical jfor, repairing TV sets ia tfi^e ' 



vacuunflube era. As a rule of thumb, replacing tube^ wouJ^^^^ir a 
malfunctioning set in about eight times out of .ten. But' a big air- 
coi^trol radar may have thousands of coro|»nents in it, with rtany of ' 
these linked together in complex w^s. Flirthermore, there is good 
reason to think that^, if a technician atferopted pimply to r^lace 
every part, he would introduce far more' troubles than he would - 

^ eliminate. Evidently, more systematic and efficient search ■ ^ ■ i 

- ■ ■ ^ 

strategies are needed. • * 

There are several ways to di^lay the relations in a systenis ■ ,^ 
^ floM charts, schematic diagrams, and so on. Hie essenti^s often boil 
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down^o-some kind" of logical graph or table. In Figure 1, a graph is 
slK3<rfn in la; it has four vertices (A, B, C, D) and ten arcs (AB, ih, 
r.etc.).. The relations in tffis graph may also be described by the* 
Boolean logic matrix in lb, or by the dictionary of successors shown 

. - / r > • 

in Ic (Kaufraann et al. , ^977) /The "successor" table reproduces the 
Boolean matrix. row by row. As shown, the vertices and arcs could be 
anything that is well-defined, and so they could represent nodes aiV3 
signal flows in an electrical network.' Many theorens have been 
derived for handling the relations in a graph or logical matrix. One 
f , ' ^® "»st important of these is that, for e^ch state of the set of 

system components, a unique structure function can be defined. Thus, 
one can calculate the influence of each component state on tKe 
overall system (Kaufmann, 1977; - Barlow, Fugsell, & Singpurwala, 
1975). As another- and moi;e sulrprising example, if a set of K "links'} 
or 3 monotone function are known, an equivalent- nej^ork can be 
obtained by placing in parallel K subnetworks; each formed of the 
carponents of a link placed in series. " The networks obtained in this 
manner can ^so be "reduced" to their simplest form. As far as we 
know, such transformations are only, employed in theoretical analysis 
of ffystenife, ^ are never utilized by working technicians, even though 
such analyses do express all the- logic in. a system. * > 
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(a) 

Figure 1. (a) Graphical r^resenfcationt ^b) boolean . 
matrix; (c) dictionary of successors. 

" Hie Boolean-table ic3ea has been extended to handle pseudo^verbal 

contentr and to .print out "sentence solutions" to conqputerized search 

problems. Pindier's Universal Verbal Puzzle Soiver ^or example, 

f * * ' 

solves any well-formed verbal membership puzzle iirtiich. can be 
expressed in terms of logical inclusion, exclusion, and ^ 
conditional ity. If ar string of problan sentences 'is inserted the- 

* * ^ i 

logical' relations from.th^ sentences ai;e stored a& a long list of . 

inclusions and exclusions. When all are entered in the confer, the 

« - .; i ■ . 

program does a brute-force recursive search until a suitable solution 
is found (Findler et aQ., 1973) . The 'sef&rGh.loop really operates upon 
the equivalent of a Boolean matrix, liiough it has c?>parently never 
been used in the troubleshooting domain, the Fira31er program would . 

< 

certainly find troubles if "enough logic from trouble-test relations 
were entered in a pseudo-verbal fomtat. The technician would eni^r 
test results in a <x>nstrain^b[iglish format, until isolation was 

attained. Another kind of "logical machine" is the. theorefn-prover 

■ •* • . 

.concept, as develoj^ by Wang and others. A theorem-prover program 
ac^pts a Cist of logical premises or conditions, llien, to search 
for an answer to a puzzle (or a trogble-shooting problem) , one asks 
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the program whether a given "theorem" (logical statement) .is valid. 

• An ingenious reduetion routine collapses the premises in such a way 
^ as to permit (h validity check* on any well-fonijed propositior^ 

involving the premises. For a verbal puz«le, a test theoren might be 
"the butler murdered the viscount", but as far. as logic is concerned, 
a theorem could be "operational amplifier S16 is malfunctioning" 
(Raphael, i976) . To be useful, both the Findlef and Wang concepts 
hav§ to be "full of data", otherwise no solution will be attained, or 
else^the logic is so loose -that a number of pseudo-solutions will be 
y printed" out, with th^ machine unable to discriminate among then. " 
In practical maintenance work, a useful format is oftto sane 
kind of a symptom-malfunction (S-M) matrix. Many projects have' 
discovered, rediscovered, and formatted the basic idea, which is 
nothing more than a table showing the dependencSs*anong symptoms and 
coTponents, An early realization of this was in^the BftMAGAT project - 
at Hv^Res'Aircraft in the 1950 's. 'The 'charts used there, called 
maintenance dependency, charts; (P©C) , showed afi ordered list of ^ 
proceduresv a rather bomplete lifting of test points and 
measurements, and also all functi<ftjal units that generate inputs, to a 

• given test.^ If a 'test failed, then- (hqpef ully) only a limited set of 
hardware signal sources had to be checked further. That basic concept 
has survived several, renamings as aiMM (Symbolic Integrated ' 
Maintenance M&nual) and PCMM (Functionaliy Oriented Maintenance 
Manual). Roeder and Ranc (1975) believe that FOm ie B»st successiful 

♦ • 

with straight-line analog functions, and that logic diagrams, state 
tables, or trees are preferred for digital circuitry. In the 
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sinplest non-sequential fonnat of S^matriXr tests are oftert listed 
' ' , as row^e K^th components or subunits in the many columns. As the 

trouble search proceeds, parts "of the table are logically eliminated, 
"and vtei one or a few units are left, those are r^ired or , ^ 
replaced. If ihe chart is correct and inclusive, and if the readings 
are correctly- mSde and. interpreted, then trouble isolation proqeeds 
. steadily.- The logie in such charts is generally easy to follow, but 
in.' the practical case. there is often ambiguity about the normality of 
a ccrplex signal or wai^eforro, and so if a mistake i& made the search 
may not converge (Joyce, 1975), Almost always, though, sope trouble 
isolation- Will be achieved. Most research. people, we believe, who 
work closely with maintenance people, in .the field have come to favor 
» aiding the technician with -some kind of S-M matiy.x approach. The 

basic idea of maintenance dependency alsd can be eqjbanced by 
. following good display an^ presentation features, as inaba (1979) and 
. ^ others have demonstrated with the PIMO format. If the system is * 

ektreroely large, then almost any format' becomes unwieldy,' Rpeder and 
• Rano (1975) estimated that ,for one s^tem,^SOTie 733 . sheets of„PCMM 
function blocic diagrams would be required td describe the system! 
■ It is best to provide ^ood S-M information to the field 
"? 'technician, and not make him generate it for fiimself . "^ The general 
rationale, for this is that wh^ working undet time and ope'r^tional 
. " ^ Stresses, the technician is not in a goed position to be discovering'- 

. logical relations. At that time, he should be ]MDa. effective 

:t» . relations alreacty discovered by himself picl by others. For 

large digital systems, there is a further rationale. Many IC chips 
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are delivered tp the system designer with data sheets which describe 
chip operation in terms, of gate equivalents. ^ But the data sheet-^iay 
-^^^ not be owiplete, or when it is encapsulated into a syst^* the unit 
can experience logical onbiguities caused by timing, noise, stray 
capacitance^ or threshold projDlems. Often the only way to separate 
all these, causes of logical failure is to do high-^ed te^ runs via 
. . autOTietic test equipment (ATE) . So in some cases the technician 

, ^ could not possibjy use logical-status derivations from a sinpl'e 
• consideratiOT of the gating logic. 

Probabilistic Models . 
't ' * . Many aspects of ssystems are probabilistic in character*. Life . 

< . . > ' failure ex^ctatioris for- components, f6r 'subsystems,, and for the * 

whole 'system can be expressed in terms _of probabij^ity distributions. 

There are also distributions for tlie ti^ it* taK^s to set up and .make 

I - . * . - f ■ • f . 

differferft Kest readings, for the liKelihopd' that a p^tticuiar failure 
■ p. ■ If ill be detected when it occurs, for the chance that a troObleshooter 
, . will introduce a. new trouble as he works on the prime . equajpinehtr and 
' . so <m. As all these distributions can change during the course Of a 
. search, aiial3f!gt& have studied ways of structuring and optimizing the 
. process, Pow^ful' solution techniques such as linear prograraning and 

" dynamic pfogjramming cah be brought to bear. 

t • • 

A simple probabilistic exan^le will give seme of the flavor. 

* Consider eac^ test or checjc reading as a means of reducing 

^ - ■" • 

uncertainty concerning the location of a fault. In the language of 

information theory (Guiasur 19771 the anount of uncertainty resolved 

by a test t is given by 
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W (t) = -Zp (s^) 1092 p (s^), ' ' ^' 



which is the sum, t>ver all sets of modules' into which the te^st 

divides th^ systan z (t) , of the probability, of the fault beingX 

within the .set, p (s. ) , times the logarithm to base two of this 

probability. Each test is then conceived as dividing the systew into \ 

two subsets (the fault beipg in one or the other) ^ The probabilities 

in the i^bve ejqpression are conditional upon all test information 

obtained up to any point in a given test sequence. ' ^ 

A rational or optimai technician might then perform next that 

•* - 
.test which resolves the greatest amount of uncertainty, per time 

spent testing. Thus, he should maximize<^ 

D (t),= W (t) , whei*e t (t) is the I 



time required to perform the test, starting at whatever system. and 
test conditicm prevails jast then. 

. . Tc^'s dynamic programming (DP) formulation (Rigney and Towne, 
1977) utilzes the structure of . tha-^stem to yield deterministic 
relationships between indicators (including test points) and the 
elements which they monitor. Time costs for extracting infontiat4@n * 
from each indicator are then either estimated ^or computed. These 
might include. tiroes for test equiprent setup, partial disassembly, or 
even wadking to another location. Finally, reliabilities are 
associated with each hafdware element. Itie DP algorithm then 
successively ocaipites the optimum next "indicator" to sample, in 
order to minimize expected 4^4^) ' dost. Itiis technique would be 

" ■ - 4lt 
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con^ider^ probabilistjic, s^ince the reliability data are 

probabilistic. \ . ' ■ ' 

Guiasu (1977) gives a similar model of classifying a systan Cin 

our doi^in, classifying a system would identify the -troubj-e* state, . 

, i^., locate the trouble). His model calculates'the entifopi' of each 

test, weights it by the cost of a test (again, this ooi^d be the time 

or inconvenience .required to perf onrT it) , and then selects the test 

that has the maximum weighted entropy. Thi^ schsne always leads to* 

the trouble, with the smallest expected "cost, if checks involve 

extraordinary amounts of ^tinie* or risk to set up, such an analysis^ 

might be quite practical. 'C^e application used'tHe. model to sel^t« ^ 

which of the 27 possible clinical tests should be mada^next on a 

human patient, to diagnose any of -13 renal diseases.' Psing the model, * 
« * . 

an^avprag^ of about eight tests are required, to conplete the ' * 

diagnosis, instead of doing the full set of 27 tests. 

-,/ * * 

Models like these are certainly "elegant rationales for test 
selection, v*iich can be valu^le for judging -the. effectiveness of a 
particular strategy and for deveXc^>ing procedural guides. It is 
unlikely, however, that a technician could employ these techniques to 

, generate his own search process. Biere may be some rough 'heuristics 
that can be ^plied by technicians, which gome out of such models. 
Hie information theory approach suggests, for example, that if all 
tests are reasonably dose to splitting the ranaining sets of 
candidates into even halves, the neift test should be the one that 
takes the. shortest time. Also, if all tests are long shots (less 

. than 5 percjent chance of finding the fault), then the test that takes 
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the least time often should be selected. SUch rules might already be 

A ^ 

known fairly well by the working technicianr ♦ » 

/ . Human" 2£3£^ Behavior • % - ' ' * 

As mentioned earlier^ it has been known for twenty-five J^ears 

A. 

that a symbolic format can* capture much of troubleshooting behavior, 

and can get at the crux of the search task. This was shown with 

various kinds of devices: tab tests, n^ortite boards with little 

windows, punchboards, and special paper formats (Grings, 1953; Glaser 

et al., 1952; Fattu, 1956). -To be sure, the synthetic format leaves 

out such key behaviors as locating test points ^ connecting meters, 

and being generally careful and*sc^e around dangerous hardware, . - 

Unless the technician ^:an troubleshoot effectively at the. symbolic - 

level, however, it would be of little avail to lae skilled, in these 

Other aspects of troubleshooting. Most iflvestigal;Qrs have therefore 

* » 

usfed some kind of synthetic format for -theit troubleshoot irig studies. 

Ntearly every study of troubleshooting behavior has noticed great 

variation in the search sequence^'iSken by technicians. Even for 

\ - 
rather sinple electronics circuits,, exact replications of paths are ^ 

rarely found. Wh6n detailed records of performances are examined no 

more than about half of the moves "J^e sense" in jcetrospect (feryan 

et al/, 1956) . Why do we «e there such marked individual 

differences, so much uninterpretable behavior, and so much^^rent 

"wandering?" Th^ best answer, we believe, is that ordinary ^ 

technicians differ Mn their knowledge about the . normal system and in 

the significance ' th^ attach to reaidings in the abnormal system. ' . 

Ttiey sinply do not share the same information bases, despite 
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similarities of training and experience. This can be. quickly 
demonstrated at any military installation by presenting a few 
technicians with a standard transistor amplifier schematic, and 
asking them separately what changes in the currents and voltages — 
wouldr result f ran parametric changes in one of the parts, such as a 
resistor that increased in value. There will not be much agreement, 
. and many mistakes will be noted. 

It is like^^^^en, that most technicians simply do not remember 
muph of the electronicjs theory that has been presented to then in 
schools and^tooks. study done in the eairly 1960^8 at a NStvy 
repair facility presented simple test items invol^g the "fifteen 
basiic vacuum-tube circuits'* to a sample of civilian shipyard 
technicians, ^along with seme elementary ele^rbnics theory such as 
KircHhoff 's Laws. The goal was to find out which circuit principles 
were roost 'difficult for the men, and how these cor?telated with 
knowledge of basic theory. All the scores were io low^that the 
research plann^ on those subjects could, not proceed. In fact,- it was 
unclear how. these men^ oould- understand, or oould repair, the oonplex 
radars and radios they work©? on.- - One 'possibility is that^lthough 

tfey could not manage detailed circuit ariiaiysis,. they could perfonti a 

> "^'^ ■ ■■ ■ 

rather- cut-and-drled overhaul and checkout procedure. When 

discussing these results, the supervisors ei^dained that detaUed 

circuit-analyzing skills w^r« useful for designers, but not 

necessarily required for maintenance sh6p people. A stmervisor would 

pften say, ?I don't know that kind of stufjf nyself, now." iventy 

years ago, Williams and Hhitsnore *(1959) showed that, whereas 



knowledge of electronics theory about the Nike*-Ajax n4ssilea ^stem 
Vras highest at the end of the training school, ability to finci faults 
was poor. Ihen, fault-finding ability iji$>roved with field experience^ 
but knowle^e of theory declined. Shepherd et al, (1977) think th^t 
it is possible that technicians may develc^ rough and effective 
search rules during the field work (and may attempt to use theory 
while ^preciating these rules) , but Once the' rules are in hand, 
theory can be forgotten. Foley's (1974) review suggested that the 
correlation tjetween theory test scores and performance is low, as is 
the relation -oetween school marks ar^ j^b performance. Thus, 
"teaching theory" is doubly ineffective. People do not learn or 
retain it well, 'and even \f they did, such knowledge is^not strongly 
tied to maintenance, effectiveness, . • 

Perha]^ the most famous behavioral analsyis of troubleshooting 
vuas Miller "^s investigation of the "half-split" technique (Miller et 

al, 1953) . The half-split procedure recomnends that a chain of . 

> 

systen elements be checked first in the middle of the chain, or in 
such a way that the two groups of elenients on both, sides are equally 

likely to contain the faulty elaneot. Th^ half-split idea has been » 

■'• ^ 

extended and generali'^ed in various ways^ to permit inclusion of 

parallel paths, prbbability-of-failure data, and cost figures. While 

•* ■■ . 

pe<^e can lear4,the half-split idea (Banncxn et al., 1967) , it is 
difficult to employ except in the ifiosV crude and obvious decision 
situations, such ^ a string of lights. In difficult, real-world 
problems, units and subsystems of widely d^ffqrent reliabilities are 
arranged in ooniplex^and non-linear chains, with obscure barriers and 
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feedbacks between links in the chain. Otherwise, tcoiileshooting 
would be rapid and conclusive by any reasonable method, and the . 
slight increases in efficiency caused by half-sf>litting would be ' * 
negli^i^le. , , - ' ' 

Other optimal search models can also be "formulated, and their 
troubleshooting effectiveness can be oonpared with actual human 
behavior. As an example of thi6, a computerized Bayesian search 
policy was defined for a certain symptan-malf unction matrix (Bond & 
Rigney, ld66) . The *con?)uter program c^rated on the conditionals that 
obtained between test symptcjn© and componeht parts. When a 
malfunction was introduced, the program would always find the trouble 
in the smallest nuniber *of steps or checks (if the trouble could be 
-found from the data .given). By updating the failur^likelihood i ' ' 
estimates for each ocHiiponent after each test, the program acted as a 
good statistician would, if. confronted with a fault-location 
problem. When a human technician's search record is dorapared to the 
performance of such a program, .it is always found that the human is 
not an qptimal troubleshooter. He makes redundant checks, forgets 
things he has already done, replaces orarponents \^ich logically ' 
cannot be "faulty,, and otten seans to mill airound the prime equipment 
in a near-aimless way. One study estimated human, search efficiency 
at about thiry percent of the qotiroal (FGigney, 1968) t Such 
qbservatic»)s might suggest that technicians are illogical or feckless 
• in their work, \Aiicb is not so. situation resembles that >*ich 
is found in the psychology of reasoning. When intelligent people are given 

syllogistic probleftis to golve, many errors and incorrect inferences 

. I ■ 
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will be found. On clbse examina^on, however r nany o£ these mistakes 
ar^ largely due to the artificiality of the syllogism, and to the 
subject's unfamiliarity with the way , that English terms like "or" and 
"some" are used in logic (Johnson, 1972). When corrected and 
practiced, and especially when syllogisms are cast in a familiar or 
** real context, subjects are indeed quite logical. Going back to the 
trouble^ooting context, the frequent near randaimess of a search 
pattern is largely due to a vague and inconplete conception of the 
basic system, and of the measurements made upon that system* An ^ 
additional bit of evidence on this point comes from the Bayesian 
fault-locator study mentioned above (Rigney, 1968) . In an extension 
of that research, technicians were required to construct their 0%*^ 
subjective symptom-malfunction matrices. To do this, they entered 
their estimates of the relationship between a reading at a test point 
and a possible malfunction in each compc»ient cell. Then, taking 
these subjective estimates as probabilities, the Bayesian comparison 
routine was run again. It was learned that v^ile most technicians are 
not "good Bayesians," even on their own subjective matrices, their 
\ ^ search does, proceed rather well, if inefficiently, and would locate ^ 

troubles eventually^ ii Jtbs matrices ^SL& fiOUSCt fiOQUgh. Again the 
conclusion is rather strong: It, is the information base, rather than 
the search algorithm, or search efficiency, that best preditts success. 

For an overall picture of technician activities. Table 1 gives a 
breakdown from one set of experimental observations (Rigney et al., 
^. 19^8) • The table is fairly discouraging. Wmt of the time (58.7%) , 
these troubleshooters did not have the malfunctioning unit included 
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in their set of "possibles." About 10 per cent of the time, the 
©juipnent was classified as fully operational, even though the 
malfunction persisted; and about 13 percent .of the time the trouble . 
already had fcen repaired, but the technicians didn't recognize that 
fact. 



Table 1 

Technician 's-'State TotalTime I of Total Time 

(min.) 

Perfect hypothesis set * 52.3 1.3 

Malfunction properly suspected, 

but hypotheses imperfect 666.3 16.5 

Malfunction not included in 

hypotheses . , 2,278.8 ' 58.7 

Bguipn^t judged all right * . \ 

despite, malfunction 405.0 10 .0 

Malfunction actually r'^ired ' ' 

but not known . 546.0 . 13.5 

A recent program of research by Rouse (1978) has been directed to 
hianan search behavior in digital networks. Itie subject faces an 
arrangeraent of AND gates, some of *Aich are good, and seme of which 
ate not operating. As in the Bayesian search comparison mentioned ■ 
previously, subjects did not perform optimally, but they were superior 
^ a brute^force searching scheme, and they could be aided by special 
softairare. When network size was increased, per;foniiance deviated 
further from optimal. Two incidental results frcm Rouse's work are 
Ihtriguing to the psychologist. One of these fihdings was that self- 
^oed initial training can transfer positively -to a forced-paced test 
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Situation. Another discovery was that thb unaided/ subjects did not 
seem to appreciate the diagnostic value a "good** test at a node 
(node not failed) . This observation is similar to Wason & Laird* s 
evidence on the idq^ tollens fallacy in hunian Reasoning, v^erein 
subjects also did not realize the meaning of information that d^d not 
directly support the hypo\±esis under review (Wason & Laird, 1^68), 



"In our fault diagnostic context^iiis idea means that humans 
have difficulty in effectively utilizing a test that indicates . 
som^ coirponent has not failed (Rouse, 1978) • 

Ihis is one instance where a troubleshooting experiment can be inte- 

♦ 

grated nicely with cognitive psychology. 

Raanussen and Jensen tl974) watched Danish technicians working on 
real equipment in atomic power plants. They noticed that technicians 
had a tendency to "do something," rather than to "reason it out," 
hence troubleshooting performances often look busy but inefficient, 
when gauged by optimal* test standards. As many- other researchers 
have noted, there were frequei)|t "....rapid sequences of goo(f--bad 

> 

diecks on actual signals against normal signals." ^fost of the 
observed perfonqsuices could be classified as being either 
typographic, functional search, or evaluation of a fault. In a 
topographic troubleshooting attempt, the technician may "lose" a good 
signal in, say, an anqplifier. 'Rie amplifier itself, and the paths in 
and out of it, would then be explored. Functional^ search proceeds on 
a "failed function" basis. Thus, if a IV picture is low, one might 
consider first the circuits that control the vertical deflection 
plates. A person who' evaluates a fault derives information from the 
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If'- 

fault it. f, without explicit localisation, A 'fuzzy^ video signal is 

... 

often due to ripple in. the main sujsply, and so the voltage regulators 
may not be doing their job, when this synptcro appeals. 

If one of Rasmussen's techhiciahs i^as stun^Sed, he might -pause, 
and then during the break in performance have seme, fresh , ideas atSout 
things to do^ At- several places in the analyses, the disparity 
between -engineering thinking" and technician wotk?3nethods was 
e\iident. The engineer often wants an intellectual understanding of 
the trouble, and a, rationale for its associated parametric changes 
throughout the hardwar-e. Cn the other hand, the technician's ^bal is 
to find iius trouble, and considerations of the logic or elegance of 
the soluti<^ a're secondary. 'Hiis is not to say that techhicians 
cannot admire an ingenious search scheme, but their focus ig usually 
quite practical, and redundancy ^s not bother than at all. 

Wescourt and Hemphill (1978) studied computer program debugging 
in both expert and novice programmers. The ej^iert may indeed have a 
better general strategy than the novice, but his roost important . • 
capability is ^ecialized memory fo^Eertain program l?ehavior 
characteristics, that is, he has a large "library" of experience which 
often enables him to bypass the usual deductive processes, A novel 
feature of the Wescouft investigation was the explicit recognition 
that a^anced programners often knowingly vzite ihobrrect prograns, 
finding it easier to debug than, to perform the logical analysis 
neoessaiy for correct coding, 

A great limitation of much behavioral research in the 
electronics troubleshooting area is t^iat very sin^e circuits and 
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equipment are employed as the performance vehicles, with the 

i 

exception. of the Rasmussen stud^ and a few others, the academic 

researchers have stayed with single DC circuits. Ohm's Law, and - 

similar engineering rudiments that are easily analyzed. The people 

who have really faced up to the<big prime-equiptient problems are 

those who were under contract to produce a ^cific aid or training 

system, and to demonstrate it on an item of (^rational complexity. 

Actually, major troubles may recguire a very lengthy analysis. Anyone 

who does not believe this stould consult Cftffron's account of ap 

advanced troubleshooting problem in a system made up of standard IC 

chips.' Seme thirty pages of techniced discussion 

explicate the test rationale (Coff ron, 1979) . Even a procedural 

guidef to this task would take several pieces of paper. 

Another big gap in behavioral research on troubleshooting is the 

« ./ 

near-absence of data on just^ how the technician codes, remembers, and 

4, • 

•> 

retrieves information about the prime equipment. Researchers often 

< 

focus too much on the outc^ie of the performance, such as the time to 
solution, or success or failure in finding a trouble, wh^ they 
should be watching the process. Electronics material is abstract, 
and that is one reason why it is so difficult to learn. What imagery 
and analogies can be used in visualizing thel relationships in an AC 
circuit? There are seme hints from mnemortics, and we knew that 
making abstract material meaningful, or coding itans into a story 
line can facilitate memory, but do subjects use such techniques? And 
could they easily learn to use them? We . know that years of intensive 
practice, say in engineering school, can produce great fluency in 
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certain basic circuit relations, but we have few tricks to facilitate 

rt-cut such long exposure. There are leariiing. difficulties, 

too, in going from qualitative tt>" quantitative ^preciation of ^ 

relations. SuRJose you are' displaying the several -currents and 

' voltages in an amplifier stage. Each little current can be s^own iiva 

. different color, and these currents can then be discussed separately 

under the operating conditions of the circuit, as in the ronarkable 

Adams series of textbooks (Adams, 1966) . If a component has changed 

slightly in valuej-though, the troubleshooter eventually may have to 

perform quantitative analysis ; perh^s yia sets of equations. How 

does this bridging from the qualitative to tKe quantitative come 

about? Again, what imagery can support the performer? In other 

settings, Bruner (1965) has shown how ingenious and concrete imagery 

schemes can make abstract and difficult concepts accessible to 

children. For instance, children can appreciate the concept of .the 

energy quantum. We need soni^ bf this ingenuity to suj^ort troubleshooter 

cognition. Ihere is some evidence that the ability„ to bring up vivid 

voluntary images can aid in problem solving, through its association 

with "independence of judgement" (Richardson, 1969) . A related 

cognitive hint comes from Rowan (1965) , who tested a programmatic * 

ajproach to solving algebra word problems: 

"...Bie program 'translates* thfe word problems, step by step 
into algebraic equations. A nuftber of algebra students do 
the same. But other youngsters first translate the English 
prose into a 'picture* of the physical situation and then 
translate this r^resentation into equations. And those who 
perform thir. indirect rather than direct translation prove 
to te more powerful problem solvers." ; 

This conclusion might apply directly to the circuit analysis domain; - 
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good circuit analysts should be those who can originate a clear picture 
of circuit actions. 

When subjects are presented with a logically simple trouble- 
shooting problem, and can approach it from (either a "Vork saving" or 
"probability" standpoint, they initially perform easy stage ghecks, 
gradually shifting to the more jjikely but more difficult kinds of 
testing. At least they did so in the Detambel and Stolurow (1,957) 
experiment, when work was represented by the number of screws in the 
lid of each component, and probability was the likelihood of a 
malfunction. As in the other work cited, there were few* optimal 
troubleshooters. A' similar setup, but with more units, was used by 
Dale (1959). His display had nineteen electric sockets, one of which 
was defective. A^ar as the subjects knew, there was no particular 
arrangement between the sockets; About a third searched randomly, 
but sane used a stepwise pattern. When the subjects were told that 
the current was flowing through the niJieteen stages, and that the 
malfunction could interrupt the current flow at any point, subjects 
were more systematic. As expected, the more informed subjects were • 
more efficient. 

Neimark (1967) used a problem boa^d equipped with shutters which 
could control viewing of test information. The optimal half-split 
strategy showed up again here, although few people- ^plied it 
strictly. Some subjects uncovered good information on the first 
look, and therefore tended to beooitie "gamblers." These people would 
eventually shift to a more systematic and conservative sdmas, 
hcwever, if their gambling did not pay off. The results again 



ERIC 



36 



.inciicate a "general rationality, but not optimal ity, in search 
behavior. 

For some complex tasks, knowledge hiearachies can be defined. 
*Gagne et al,, (1961) took as the terminal behavior the capability of 

r 

finding formulas for the Surfi of n terms in a nuiriber series. This 
terminal behavior was broken down into prerequisite behaviors, and . 
these were then arranged in an ascending series, which were taught 
one level at a time. The final or highest level was not attoripted / 
until all the prerequisite behaviors were mastered. Most of^sthe / 
aipirifcal studies of this learning-hierarchy approach use "clean" ' / 
mathematical tasks-, aod in' those cases the data are predicted well 
from the stratif ied-skill arrangement (Gagne et al., (1961). While 
roost training courses do assume prerequisites and cumulative 
knowledges, a logically engineered hiearachy does not guarantee 
learning efficiency, and may even hold progress (Merrill, 1965; 
Jphnson, 1972). Subjects, can learn "from the tdp down," and can • 
roaster Xower-^evel skills while working. pn the higher levels. It is 
still true, of course, that ap some point th^ lower-level 
capabilities will have to be develc^. The learning hierarchy idea 
has not been fully been evaluated in troubleshooting of real 
equipnent, though hierarchies were defined f6r a fairly involved DC 
circuit problem (Wollroer & Bond, 1975) . 

Cognitive styles can soiietiroes predict aspects of intellectual 
perfooianoe that cannot be explained by aptitude and ej^erience. One 
style feature is the way that a problen solver gathers information 
about his laroblan. in Gaier's (1955) Air Force study, trainees who 
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preferred to seek principles rather than facts performed better in a 
mechanics course. 6ar(±ier & Schoen (1962) found a stable style of 
cognitive control in their free-sorting test situation. Saoe 
subjects made sharp and* genuine cognitive differentiations when 
arranging a set of items, v^ile others merely listed concrete 
features, and did little "real thinking." Interestingly, this 
"differentiating" cognitive style was independent of general 
intelligence. Perhaps it could be* confirmed in a troubleshooting 
donain, and could help to es^lain the wide individual differences in 
eventual performance, among technicians vdio are all selected on 
general intelligence. Similar expectations might apply to the Witkin 
"field indepenjience" concept, since people scoring high on that 
variable tend to achieve figure concepts readily, and" to perform well 
on the classic^ problems of Duncker and Maier .{ifevis & Klausroeier, 
1970) . Nearly |all of the cognitive style variables proposed so far 
are of questiorjable utility, insofar as reliability and' 
general izability are ooocetned, but, they offer some intriguing 
research possibilities. 

As mentioned earlier, negative information is not employed in the 
same way as positive information. Wason's (1960) r^arch is a good 
illustration. In asking college students to generate a rule from a 
number series, he instructed .,'.."You will be given three numbers 
which conform to a sinple rule that I have in mind.... Your aim is to 
discover this rule by writing down sets of l^iree numbers, together 
with reasons for your choice." A significant finding was that the 
subjects seaned to avoid negative or disoonf inning examples that 

\ 
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could eliminate a given hypothesis. Campbell's (1965) experimeht was 
along the same line. It presented word problems wherein the letters 
of the words were replaced by other letters, if the letter 
substitutibn had to be done indirectly, 'by elimination, then the 
. problems were more difficult. Johnson (1972) interprets such results 
in terms of information load: 

\» . * "* 

**.... Tliere is an URjer limit to the amount of information that^ 
the individual can handle, and th^t as he approaches this limit 
he makes errors. Presumably, he retains the information that 
first comes to him vhile^taking in new information to be 
related to it. If he can relate the new information directly 
to v^t is being retained^ soXution is easy. But if he must- 
transform the new infonnjation, as from negative to affirmative, 
or reverse the order, or change the subject, the total load may 
approach his capacity and errors may occur." 

The positive-negative logical disparity in human processing can 
also be demonstrated by timing the performance on specially- 
constructed items. The statenent "seventy-eight is an odd nuntoer" 
* can be answered quicker than the negative ooui^Tart "fifty-seven is 
y not an even number." Aa Johnson (1972) observes, such items can be 
written in con$)letion form, and again the positive statement is 
evaluated sooner, Jones (1966) showed that subjects can mark "3,4, 
7, 8" faster that they can mark "all numbers except 1, 2, 5, 6." 
Perhaps the ultimate in timing technology fii this domain was achieved 
by Trabasso (1967) . He gave one subject 32 selection rules about two- 
part-figures that were systematically varied in size (large or 
anaU) and color (orange or green) . A rule was phrased in terms of 
- . logical "ard" (conjunctive) or logical "or" (disjurictive) . A rule 
like "orange and oraAge" was verified faster (.634 sees.) than 
"orange and not-green" (.736 sees.), and "not green and not green" 
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took ,824 g^nds. Trabasso then assumed that these time differences 
were due to-a basic transformation time. Thus, "orange and not- 
green," though logically equivalent to "orange and oranger" took 
about a tenth ot a second longer because csne transformation was 
required for the not-green term. TVo transformations ("not green and 
not-green") would add another tenth of a second. Perhaps such 
transformation tiroes are basic parameters of human information 
prdcessxhg capability. As far as we know, nobody, has measured them 
in troubleshooters. 

ConditiCHial reasoning, involves some kind of if-then contin- 
gency. Here again, many logical errors are d::^rved v^en people are 
asked to process conditional statements. Ihe format structure can be 
expressed in P Q form, as follows (Johnson, 1972): 

If X is a f ish^^tjien X can swim P — Q 

X is a fish, P 
iSierefore, X can swim Q 
Nearly everybody perceives this as a valid argument. But the ^ 
follow jyng fallacy is also perceived as valid: 

If X is' a fish, then X can swim P — ^ Q 

X can swim Q - 

Ttierefore, X is a fish .'.P. 
One trouble with the conditional is that the subject may put 
ten|x>ral or causal, meaning ^into a structure which is supposed to be 
strictly logical. English wordS'like if and then haveja distinct . 
tine and order flavor about them. Also^ the same dif f milty with" 
handling negation in ordinary logic is found with conditional 
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falsity. Wasbn's (1968) demonstration involves only four single 

cards, but it shows the logical and psychological skeleton of the 

conditional. The four cards have 3, B, and- 7 on their respective; 

faces, and the subject is asked to evaluate the validity of the rple: 

if there is. a D on one side of any card, there is a 3 on the other 

side. Which caj^ds should be turned over? The correct answer is D' 

% • - 

and 7. * In the event, everybody selected D, because if there was- a 3 
on the^ other side the rule would be violated, but it was much more 
RffiCult for people to see that 7 was the only other choice that 
cdlsld invalidate the rule. Many people turn over the 3 card, and 
even 4f ter hearing the answer and having it explained, > some stibjects 
do not feel right about it. Johnson-Laird and WasVn (1970) think' * 
that the difficulty stems from the incomplete "truth tables" which " ' 
are* held by most peoplg, and fret their set for truth rather than for 
ialsity. If the arguments are presented in a "practical setting," 

say with real objects, then the logical errors decrease sharply. As 

• ' i 

far as we know, there have been no systematic studies of the 
"negative logic" in human troubleshooting. 

•fliis review, brief as it is, has given a gliirpse of behavioral - 
research that is relevant to troubleshooting. T&ken together, peqple 
who work on troubleshooting programs and devices seem never .to refer 
to it^ Humrro*s conclusion iroro over twenty years ago still holds: 
"....the study of human problem- solving, concept fpnnation, 
probability learning, and so forth, has provided little data of 
iniaediate use to the researcher in electroiic troid^lesihooting, but 
has provided oertain limits as to important variables...." (Czeh, 
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1957) . Rather little of this work has influenced the practical design 
of ecpjipnents, courses, and troubleshooting prbcedures^ Occasicxjally 
one viU find an engineer v*io has heard of the half-i^lit. notion, and 
some cjonception of a learning hierarchy is discernible in many lesson 
plans,^ But such itens as "ronefnonics for electronics," "story coding" 
• for iienory, cognitive" sytle, and hun^ preceding of conditional ity 
and falsification,^ are apparently not recognized outside of 
psychology. 'This could be due simply t;o interdisciplinary ignorance, 
but then it is very difficult to take a corlception such as cognitive 
style, or "imagery-supported electronics," and to develop it into 
-something useful for the real troubleshooting dranain. Si^jpose one 
wished to 'match a military troubleshooting aid or training device toj 
an individual's cognitive style. There would have t^o be a lot of 
psychometric work, difficult field trials, and long-term coirtnitment 

IT 

from operational coirenands. Instead of going through all this, the 

4 * 

practical operator might ask his technologists to set up the ^ 
troubleshooting task so as to reduce the iirpact of cognitive style on 
the search task, iliat is exactly what the aiders have done. If the 
c& and suRwrts are effective, then individual va'riance in 
processing skill will be minimized, or at least reduced. We have 
already noted that "concretizing" the P»s and Q's of logic problons 
will tend to. reduce the. logical errors, to ini>rove performance 
generally ,,gipnd to make the, task more routine for everybody. 'The 
practical training and operational executive wants this kind of 
aiding, rather than an academic explication of the factors underlying 
the performance when the aiding is no^t present. ' 

> ■ ' - 
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Digital systems bring special problems and opportunitiev to the 
behavior analyst. A large digital systen' is an assenbly of .xiany 
copies of a few standard units. There are only about a dozen basic 
logic functions such as AND, OR, NOR, or NM®. Each of these 
functions has a logical truth table, and can be realized in hardware 
by means: of various technologies. Usually an itan such as an m gate 
is encapsulated as a small integrated circuit. Operating 
■ sectiorts of a system can then be put together by an array of the 

information which is coded in .four 
binary digits as 0'^ and I's can be converted to decijnal form by 
means of 8 inverters and 8 NAND gates, and the whole converter can be 
smaller than a letter, or even a dot, on this page (Coffron, 1979) . 

understanding a digital system seems to be a psychological ' 
challenge for many people, it takes sane time to get used to the 
many logical O's and Vs that are processed and transferred about. 
Sitice transistors, resistors, and diodes are often anplcyed in. the 
circuits, the student also must know .enough electronics to folldw the 
current states in a bistable device or an operaUonal amplifier. 
Often, there is not much imagery to assist human memory in holding, 
the big logic arrays mind. The digital system with its abstract 
desigr\ may seem to be austere and counter-intuitive. 

Digital ooirponents such as hunters and shift registers are 
ariven by special clocks, and so the timing is extremely critical • 
clean and definite natufie df the signals, and of the flip-flop 
cirailts, means that a timing 'diagram x>f signals and events in a 
circuit can be very precise. A timing layout may show free-running 
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clock pulses running along a top line, with real time going to the 

right. Belcw this there could be many other event lines, which show 

preset, clear, and output-synchronous ev^ts. Such diagrams seem not 

to be very memorable, however, and the meaning o£ all the time 

intervals can quickly be forgotten. Furthermore, unless every event 

is understood at some physical action level, the diagram may be 

useless in a real trouble search, and the technician can get lost in 

a jumble of microsecond-duration events. The cognitive problem in digital 

electronics seems to be somewhat similar to that of the computer 

progr airier (6r d^Dt^ger) who must at once understand a oon^dex program 

at a global Itevel, yet be able to deal with the most minute aspect of 

one or more individual instructions. 

■Riere is another type of intellectual effort that is required of 

aoy electronic .technician, and particularly so of one working on 

a digital system. A quote from an engineering text will give the 

flavor; ^ . 

"....When we look at the overall system.... we see feed-back * 
lo(^, DAC's, LED displays, and digital counters.... Hie 
troubleshootei; must develop the ability to separate in his * 
thinking the. individual subcircuits from the oonplete 
circuit or system. iSiis is a mental separation at first for 
the purpose of understanding the small circuit and large 
system operation. The second step will be to s^ly the 
maitid separation to the (^sical hardware to accomplish the 
task of troubleshooting." (Coffron 1979). 

Again, this "mental separation" depends on a detailed 
appreciation of circuit functions and inter-relations, at 
some level. In fact, someone who could trouble^K>ot each little sub- 
circuit would probably not i^ed much further capability td do the 
whole thing. There is a big picture, of course, but working inside 
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. tjie big picture depends on many little perceptions which are . 
accurate. One training strategy, wtich is often used in the computer 
industry, is to teach just enough electronic theory* for the 
technician to understand .the dozen basic logic funtions in digital 
processors, or perh^s only that set that he "has to know about," in 
order to use the aids furnished to him. ^ * 

•mere are special problems in the location of intermittent 
faults, and the process of finding them is just, beginning to^ 
modeled (Varshney, 1978) . Many such faults emerge only under certain 
conditions of age, timing, and adverse operation. Thp necessity for 
finding intenfiittents. is one of the most challenging problems the 
technician faces now, and this^will probably continue into the 
future. From theoretical* considerations, .the "memory" required for 
intermittent troubles will be found partly in the technician's head, 

and partly in his aids. No behavioral research has been done in this 

< 

^^rea. 

If the troubleshooter of the ininediate future will indeed be 
primarily a highly-skilled user of complex procedures and aids, then 
information about the way that procedures are leamid should be useful 
in preparing such people. Again, there are promising ideas and 
hints from academic research, but not much that is easily 
Norman (1978) , for exanqple, says that the common sense notion of ^ 
repeats practice as the k^ to learning may be largely erroreous. 
instead of endless repetition, learning often proceeds better if a 
story line is mapped onto the new material, or if imaiKmics and 
visualization can be utilized. A well-learned ccnplex procedure can 
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give the perf onner *a play-like experience of "flow, " which can be 

intensely satisfying (Csikszentmihalyi, 1975) . Norman believes that a 

student should know more about* his own psychology of learning, and 

about the way he responds to practice schedules, feedback, and the 

arranganent of materials. White (1959) points out that being able to 

do things well gives one a feeling of "effectance," gi being in 

control of one's actions. Since White's effectance idea requires 
p * 

that the performer have some opportunity for originating parts of a 
task sequence, the designers of future trouble-shooting aids might 
well program certain possibilities for individual choice into their 
coirplex test routines. The best troubleshooting aid might be one 
which has just the pfght amount of cognitive novelty for the person 
using it! 

It is well known that there aire ways to memorize very large 
quantities of data by utilizing "peg-wofd hooks" as in the Fur^t 
memory system (Loganv 1956) . Even if a large table of symptom^ 
function relationships were memorized, howe^>er, the .problem of 
correctly exploiting thoe data would remain. Considering the ease of 
substituting paper or microcircuitry for human mcnior^, the pursuit of 
•memory enhancement, without understanding, seems unattractive. 
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#. fiy^qj.n?gffing h Maintalp^ ^]^ System * 
People who have worked on human factors problans in coiplex ' 
equipnent often have disparate views of the world, but on one issue* 
^ they are almost universally agreed: almost never is sufficient . 

« 

attention given to -people problems" in the design of a new prime • 

c*>servation will suggest whj' this 

^ is so. In the standard R .& d scenario, a scientist or engineer has 
a bright idea, for a new. hardware iten. After seme ooiipany-funded 
preliminary work a proposal is made to NASA, F^, or a inilitary , 
service. Contracts are let, and the develc^nt cycle begins, in 
the early stages, nearly the total em^iasis is on engineering 
feasibility, and such matters as packaging, logistics, and ease of 
maintenance always "can be taken care of later." Since the eqi^pnent 
is complex and is challenging the technical limits, there-.^ll be 
mahy probleros.in getting out a working prototype, Imd -fcy there 
^ probably wili be time and cost overruns. If the hardware conception 
is basically valid, then a short time before delivery some attention 

wUl be given to support functions, and the maintainability 
^ialist and other support peqple will be allowed to look at the 

project. By then, though, radical design changes may be expensive or 

iii?»ssible, and quick fixes may be attempted. Once more, an 

incomplete system goes into service. 

df course, there can be strong persons along the develqnent 
, trail «to insist on maintenance csonsid^ration^ all the way; At 

mUitary procurement desl^s these are often ^people who have had to use 

new equipment in the l^eld, and who are sensitl^ye-to the problans. 
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But the perscxns with the technological ideas are' usual^f^ engineers ^ 
and scientists who perceive the main chaXl^e as one of innovative 
design, "^ey get, their satisfactions, and their reputations, from 
the design activities.^ The program managers, on the other hand, have 
budget and delivery schedules in mind, as well as acceptance tests 
and field trials. A lot of money and prestige is riding on the bapic 
project fieasibility. In these circumstances,, maintainability is just 
another secondary support irritation to worry about. 

There is a "design side" to maintainability analysis, and 
there is a "system availability" side. Availability is often 
expressed as a joint measure of reliability and. maintainability. 
Thus, if reliability is denoted by Mean Time Between Failure (MTBF) 
and if maintainabHity is measured as Mean Tijne To Repair (MWTR) , 
then Availability (A) can be expressed as a sinple ratio: 

m£ 



' A « MIBF + MnSt 
Formulas like this can be computed, graphed, and extended 
mathematically in many ways. As just one example, MPm is often 
assumed to follow a log-normal distribution, and under this 
assunption the likelihoods of certain availability levels mey be 
coir|5uted. EJquations like this can show clearly the tradeoff between 
reliability and maintainability, and it is no doubt interesting to 
work out gr^hs relating availability to various nupnbers of 
R&D dollars. But such fonmil^ are rather empty, because they do 
not tell J;;q ^ in brder to achieve a given level of availability. 
Hence we do not pursue the availability question here, exc^ to say 
that we have almost never seen a military service hold a manufacturer 
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to the availability promised in the original proposal. . 

•nie design side of maintainability engineering is a inuch richer 
and more useful discipline, it encourages the formulation of an 
explicit, maintenance policy, it gives specific design rules vrtiich are 
(sometimes) behaviorally based, and it can stay close to the work 
that must be perf onned. When fully, iinplemented, a maintainability 
effort starts with the maintenance concept for a given prime 
equipnent. The concept defines levels of maintenance, the 
arranganents for support by mobile and depot facUities, the nuiriaer 
of operational * and ro^ntenance installations, spare parts facilities, 
and other system requirements. There will also be a repair policy. 
A system can be designed to Jbe nonmaintainable, partially r^irable, 
or fully repairable (a nonmaintainable iten is not repaired, but 
simply replaced by a spare) . A partially r^irable systen will have 
sane assemblies which are throwaway items, v*iiie others will be fixed 
when they fail, ideally the maintenance concept will be complete, 
that is, actual behavior constraints required at the different levels 
wUl be fully designated. 

Subsequent st^s in a good roatintainability program will go on to 
tradeoffsT-designjssistance and review, and the demonstration of a 
maintenance capability (Blanchard & Lowery, 1969) . Check-lists and 
hardware evaluaUon sheets ai;e' used to score each equipment item, in 
a large project, there will be much paper related to design 
reoonmendations, and regular maintainabUity meetings* The final 
danohstration is often two-phased, nje first one takes place at the 
contractor's facility and uses contractor personnel? the second is 
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held in the user's environment such as Navy ship,* with the user's 
people in charge. 

It is surprisingly difficult to £!CfiK£ the- utility of a 
maintainability program. There are many case studies and 
illustrations of the presumed benefits of redesigning an equipnent 
for ease of maintenance. A favorite textbook exanple is an Amy 
field radio, which had many arbitrarily designed and located switches 
and dials areJ contrdls on its front panel. To operate or check the 
original radio, a technician had to explore the entire panel. Otie 
task sequence was roac^ more regular, allowing the ^operator to start 
at the top and work clockwise around the front panel. Readout dials 
and test points were made more legible, and the switches were 
^enlarged and sh^d so they could be discriminated by a gloved hand 
^In^the dark. Inside the box, a partial rearrangement of wires and 
parts was done, so that the electronic states were more clear-cut to 
the repairman, and use of clips and clamps hastened the renoval of 
parts. Yet, nobody ever really proved these inprovements resulted in 
better field usefulness of that radio. 

Maintainability design and planning effort should be considered 
as a necessary but insufficient condition for good troubleshooting. 
Ihe. cognitive load on the technician is the key thing. li does 
not help the techni^an much toliave an accessible test point if the 
meaning of the Voltage at that place is obscure, or if he does not 
know v*jich voltage to look at. We would argue for a basic behavioral 
reference for many of the design projoosals made by maintainability 
people. ^ 
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MaihtainabHity programs rarely admit a fact which is obvious to 
others: sonetimes the corrective maintenance technician has an' 
iinpossible task. One Navy air-search radar, for example, could not 
be maintained by typical technicians. The syston was extremely 
conplicated. Designed just before the integrated-circuit 
technological revolution, it had several cabinets jammed full of 
subsystens. Sane of the test sets acooirpanying it were novel, and 
difficult to use. The manuals were beautifully bound but 
ijnpenetrable to the average techfiician. Spares were often very 
expensive, which made a throwaway policy untenable. A graduate 
electrical engineer might have kept the radar going with months of 
specific training on -it, no other duties to perform, and a b^g parts 
budget. Here was a total operational failure, yet the 
maintairjability documents and field advisories kept coming in a 
steady^flow, both from the manufacturer and the Washington bureaus. 
Nobody ever said, out loud, that the systan was not maintainable. 
The point is obvious: real troubleshooting in the services is often 

by pec^ie who have a year or less in a military technical school 
and some months in the field, and ihere are many itens they sin¥>ly 
cannot service. Wh^ the maintainability and planning documents do 
deal with human capabUities, the reference is of ten to formal school 
levels or ratings, with assumptions of c^ability in the higher 
enlisted ranks, the assunptions are often wrong; many first class 
ET's are not first class troubleshooters. ^ • 

Is so ntich eguipnent plaljay unsatisfactory from an 
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elementary design standpoint? Do the engineers not know of our 

maintainability design guides and review procedures? What about the ' 

^ign aiggestions we have catalogued— are they seriously inadequate 

or incomplete? Maybe the best answer is that while maintainability 

guides and directives are ^f ten known by and available to the 

designer, there is not a sufficiently clear payoff to be had from 

consulting than in a systaiatic way. After all, much of the advice is 

/ * 
rather obvious: Keep it aiirple, label things clearly, organise check 

sequenc^ in a natural or<fer, write the manuals so users can 



understand than, encase f 



agile parts in plastic. Glancing through a 



set of hOTdlies like thods, a profesaifOnal engineer might think he 
was not getting anything lovel. Perhaps an even more ba%ic answer, 
and a major key to eventual utli^tion of the advice we do have, is 
that maintainability people are staff people, and seldom have the 
power t^^rpose their arts onto a design; '.TSiis situation can change 
in equipment must be maintainable, as in a submarine or a NASA 
ice labbratory. Fortunately, human factors courses ^>pear more 
frequently in engineering curricula, and are also popular in middle- 
management military courses. "Rie most critical- pressure for 
application will come from the users. Unless that pressure is 
applied, good maintainability engineering will continue to be very 
rare. 

SECTION III. AIDING AND TRAININ5 THE TBOUBLESKOOOSR 



The tech manual is the original aid. it has always been provided 
as a part of the prime equipnent package. Much effort always goes 
into it, and yet the tech manuals have usually been unsatisfactory to 
the user. Maybe this is because there are many users— maintenance 
technicians, installers, curators, parts suppliers, field engineers, 
field change people, curriculum planners, and so on. 

While the information in a manual may be correct, it may be 
inpossible for a typical user to find out just what he wants to 
know. Hie troublesh^ting s^tion of the manual was', and often still 
is, not arranged well for a miliUry technician who is trying to €ind 
and rqplace a faulty unit in the middle of the night. 

■niings are better when special troubleshooting manuals are 
produced, on the basis of behavioral considerations such as these 
(Rigney, 1970); 

1- ^4^^ g^!;^<3 ^o^^<^ the Sf>a«ence of ^PnKc 
in_the_job. The first things in the book should be the 
first things the technician does as the search proceeds* 
Hie technician works through the book, which becomes 
more detailed. . '* • 

2- T|>e yf opn^atjop g eyp tg^ ghoMld be re l e v an t to the/ioh . 
^lusn^a troubleshooter's manual should include only the 
iteans that the individual needs. 

3- ;^^.^,^f^ fl^ff!;^^^,^^^^ ^ appropriate to th6 uaer'^ 
technical kpoy^^^. a trained technician probably does 
not know much electronics theory, and he cannot use an 
engineer's course on theory of operation of the 
particular system he is working c»i. now. 

4. information should ae^^rfft? An idealized wave- 
s form in a manual may net be a good basis for cxwmrison 
with a real wave i^hape at a test point. It is 
ofta> surprisingly difficult, or iapossihler to obtain the 
^d "normal" parameters shown in a manual. Failure to 
oBtain the reference indications can cause considerahie 
confusion for the technician. 
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5, "tti^ format ^mXd be convenient to us^. in the user's 
environment. Manuals should be easy to carry, to read, 
and to pr<^ \jp in cran^ spaces. Sometimes a •manual" 
could be sonething like a pack of paastic cards riveted 
together at the comers, or a small device with slides 
and wheels. 

There are military specifications calling for such improvements, 

of course, but the manuals still are produced by much the- same 

process, and the same people, that produced the older inadequate 

ones. .DOD has devoted large sums to manual-^mprov€ment projects in 

recent years, such as "new concept" projects in manual writing and 

production .{Klesch, 1979) . ISiese efforts are beginning to have an . 

, inpact. A most convincing demonstration by Potter (1976) compared a 

fully-proceduralized (FFTA) manual ai^ a logic-tree manual aTTA) 

with a regular Technical Order (TO). Both new formats produced 

better troubleshooting, less Wasted parts, and positive acceptance by 

the field people. This study used Air Force operational radar and ' 

1 conputer gear, and presented realistic fault-location problems. 

Abou^ t^e only disadvantages of the new formats were the higher 

preparation costs, and the extreme necessity for avoiding errors in 

the do<^ims«tation Mf a man following FP^instructions is told to go 

td page 37 when he should go tp page 73, ^e may be irretrievably 

lost.) Shriver'S' analysis of tt^e new presentation formats indicates 

that they are all based on a sihgle assumption; ^ 

•....experts can analyzi the equipment, figure out what 
. shoui^be done to it imWvery possible situation, and 
record this in Technicali Manuals that even a novice can 
use correcUy" (Shriver, \l975) . 

ishriver goes' state that^this concept is realized by means 

of four techniques foj producing ctontent (task analysis, behavioral 
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cues at each step, etc:); and the ^preciation of this oontentas 
then enhanced by employing good readability and di^lay considerations. 
Shriver also observes that the technician may only need ^stem 
understanding at a block level, and not at an electron level. Trtiis*" 
. accords well with the evidertce that electron-level knowledge is not 
well retained' following academic training.* 
Hand-held Aids 

In this category* we think of items like the flow diagram for the 
. "B-? fire control" (Morgan, ^t al., 1963), and the Experimental Fault 
Locator (XFL) troubleshooting wheel, developed by Rigney and Froroer 
at WJL {Rigney, et al#, 1965). Morgan's diagram offered a clear 
functional layout of test points and line replaceable units, and also . 
furnished a sequence of checks to follow. Rough but effective test 
tolerances were also provided. For example, the guide says that at a 
testpoint the "wave shape is not critical;" but the absence of a 
signal, or the presence of spurious oscillations, does indicate a 
malfunction. This is just the kind of information that' a technician 
needs, and can use. 

The XFL vAieel is a circular plastic device, similar to a circular 
slide rule; it contains a syn^rtion-malf unction matrix for a Navy^ 
transceiver,' along with an algorithm for selecting the next front 

I 

panel test to perform until the fault is "localized. Hhe device thus 
unburdens the technician in two respects: mdtory for the malfunction- 
significant syn^ORS, and* the foimulation of a Ibgjteal search plan 
for narrowing down the fault. Radio operators, who had no training 
in inaintena|ioe work, were able to use the XFL. in a few hours, and 
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could localize faults inserted in the equipment. Ttds finding not 
only indicated tihe vali» of the aid, but also confirmed our gafteral 
thesis that, if an adequate next-step selection aid is available to a 
technician, good troubleshooting behavior c^ be expected. ' 

Many other hand-held devices like these two have been invented, 
and some have reached the fieia. The original research studies 
almost always show the effectiveness o£ such items, but scamehow the 
aids themselves rarely become widely distributed and utilized. One 
thing is certain: One cannot simply develop a device, send it to a 
Navy or .Anny f i^d unit, and as'sume it will be employed. The field 
people have to be pr^red for the device, they have to be trained in 
its application, and most injjortant of all, they have to feel a need 
for the things that the device can do. . 

Perhaps one reason that aiding devices are not widely seen on the 
job is that they have been somewhat resistant to updating. The aid 
may, for instance, not take account of field changes in the 
particular prime equipment. If the technician tries to use it, and 
encounters an error or omission, he may ^wisely) discard the whole 
thing, when actually it would wock most of the time. We have often 
heard technicians say that a new tech manual or other job aid is "no 
good^" but subsequently close questioning indicated that the item had 
never been given a fair tr^al. Sometimes an outside observer gets 
the idea that technicians in the field are too suspicious of schemes 
and devices which are provided to them. It is hard to say how much of 
this rejection is due to previdus experience with really inadequate 
materials, and how much is due to the field person's need to maintain 

ft 
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his own discretion and control over his work. What cannot 

be ignored is that sotething about the field environment presents 

serious difficulties to the successful introduction of a job aid. 

Right now, the most exciting hand-held aid possibilities are 

«< t , ^ . 

those involving micro-conpaters, especially those termed 
programmable calculators, in 1979 technology, a small mapocy cube 
{read-only manory or RDM) about a half-inch on each side, holding 
nearly a thousand program instructiorts, plugs into the side cf a 
snail hand-held oonijuter. If desired, a light printer can be 
attached, rtie whole thing weighs only a few pounds, is ultra- 
reliable, and costs less than 400 dollars. Special-purpose ROM's 
approximately the size of sugar cubes, "can be produced economically 

1 

in quantities over a few hundred. Thus, we envision a proliferation 
of this technology which transforms a genferal-purpose processor into 
a special-purpose machine. ^ 

In addition to ocaimercial applications of this technology, Ttowne 
(1977) has iirplemented a detailed troubleshooting strategy for the 
UR032 tranceiver, to demonstrate the^ potential of guiding the 
troubleshooter with a hand-held processor. The services have also 
begun to employ €his technology both in the schools and the fleet. 
The latest develc^ment . in progranmable calculators, the Hewlett ' 
Packard 41-C, provide non-volatile manory, eliminating the need to 
load a program each day. And there is a full alphanumeric display. 
Bile latter capability allows a truly interactive program in lAiich 
user inputs are ^led by natural English words and phrases. While not 
yet built into a prograwnable calculator, voice synthesizers and 
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generators have been sufficiently miniaturized and mass-produced to 
be incorporatd in the next generation of hand-held devices. 
ffi#ap-pg^ent^ SSSm^ Mife 

The term "human-oriented" is used here to differentiate such 
^sterns from automatic test equijinent, which is usually strictly 
harcVare-oriented. Until a fey years ago, "catpiter assisted** 
usually meant that an aid was driven ty its own prime equipment. 
Wfeen timeshare technology became cheap and reliable, it became 
convenient to store maintenance charts and sequence information in a 
master program which could advise the technician about the best next 
test to perform, A con?>lete system like this becomes a more or less 
sophisticated page turner, v^ich is based on a large network of 
contingent action-choice^ Though the concept seems simple enough, " 
setting up a practical program is a considerable task. A good 
example is a special logic-tree program written in the IBM . 
COursewriter language using the im 1440-1410 computer and a 1050 
student terminal for storage and display (Rigney et al., 1966). After 
only an hour or so of coflputer-aided practice, a Qii^ ' Radioman who 
had never operated the UIJC-32 trai^iver was able to work through 
five trouble sequences on the real equipment, which was standing next 
to the oomputer-teqsinal. All except one of these problems were 
"deep", in the sense that th^^sCTuired correct sequencing and 
interpretation of at least ten orliiore tests. 

Itiis demonstration showed th/t special languages like Course- ' 
writer can be utilized quickly by electronics instructors who are not 
tredned programmers, and that^en a few hours of conpiter-guided 
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practice can show interesting beha^-Qral results. For' each trouble- 
shooting problerr., however, a special program had to be written in 
otder to provide a "trace" through the synptont-malfunction tree. To . 
generalize this guidance fanction, and to make it less prc4>lem 
dependent, Towne and King of BTL produced the AIDE configuration. 
AIDE accepts troubleshooting tree data from any prime ecjuiFnent in a 
standardized fbrmat, and furnishes analyses of "next best check," 
detailed task specifications, procedure lists, exercises in symptoiT: 
interpretation, exerci^s in equipment set-up, or simulated* 
maintenance problems. The unique goal of AIDE was to serve as a I* 
performance aid,* a training syston, or some mix of those two 
functions, as detemiined from a particular user's needs aiv3 experience, 
At the moment, t^e AIDE demonstration data base stores 125 colfor 
micrographic images and thousands of words of ^propriate fault- 
* locating software. This is enough information for a significant part 
of a large radar system, for the complete radar systQfl«H)erhaps 1000 
images would be needed, but there would be na additi6nal software or 
confvter memory. 

4 

. Another advanced coriputerized aid is the UXSXm Diagnostic Test 
Set,^ vdiich was designed by DePaul and his associates, as a means of 
meeting sixteen desirable attributes of a performance aid (DePaul, 
1979) . rtie WGnx> liarch^are is about the same size as a 15-inch * 
portable W ^t. it can proivide test sequences, r^ir parts 
-listings, reliabUity data, interpretation of test readings, and so 
forth. The equivalent of 500. pages of manual information can be 
stored on d)e S-Va-illfch floppy disk, which can be accessed in 
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nisnerous^ways. 

•rtiere is surely reason for qptimism, then, on the- technology side 
of th^ aiding question. The utilization of the hand^held oonpiterr 
with a special manory cube for maintenance, is on the way. And 
slightly larger systems like AIDE and UX3!D0 can store ^ deliver an 
iintnense amount of information, and can operate in several aiding and 
'training modes*. TlTese e^roaches can realiz^ all the behavioral ' 
^^f4eieneies promised by the fully-procedural ized manuals, in a more 
^ responsive format. ^ 

o 

certain problems in getting all this to the user will persist, 
however. Over the near term, success or failure of the aiding 
technology may depend heavily on the military's reaction to soft^er 
problems of implementation, which are intriiisic to all iarge 

* organizations. Suppose a new aiding idea is developed, tested, and 
rea(^ to be put into the field. Nobody greets it with open arms. - 
The military field ocaimander has long since adapted to marginal 
performsunce as the standard, and he knows tKat he will b^ gone before 
any perceptible benefits will come from this new idea. There are 

- large and unwieldy support bureaus who might, be threatened by the new 

* idea. The training planners, lesson-writers, and school managers 
usually will not be eager to help; they may fear that the aid will 

^make their schools apear to be outmoded or unnecessary. Hiese are 
conservative, entrenched, and powerful agencies; they have their own 
d^artnents for research and develoanent, which it is in their 
ii^erest to preserve and expand. Ttis^ will say, "Well, we're already 
^^^ing that." They will think the idea is intruding upon their 
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doiain, or they will want it to be implanented, if at ail, "by a 
section of their present organization* at a time convenient for 
them. Or, theS^ nay attanpt to offer a "Vatered down- version (Foley, 
1978) . Ttiey may claim that introduction of the new idea will cause 
budget difficulties, or they may prevent access to research subjects 
in training schools, because %...we have personnel quotas to 
deliver." i 

Given such a conservative systan, what can be done by the 
research community? One thing we can do is to stay closer to the 
real (^rational needs, and to make our defnonstpations sharper, so 
that the value of new methods is clearly cost-effective. Training 
establishments and manual writers often concentrate on delivering 
manpower quotas and staying within budgets. ' They are not necessarily 
reinforced for solving the f Jk^ld^oblems. inaba (1979) put it 
plainly enough: 

» * . » 

"Lil^e attention is given to the usability of the 
manuals or the performance capabilities of the 
students.... a change is made and the technicians become 
more productive, the publications and/or training manager , 
06 not share in the benefits— except possibly for a 
conpliment. " 

♦ 

But this means that sonebody who concentrate on the ^eal 
operational problans may find that he can make an inpact outside 
the usual bureaucracy. A strong project office, for example, is 
often able to inplanent quicKly a new aiding notion, as part of a 
harArarfe procurement package. The maintenance concept for at least 
one Navy shipboard oonputer systeni was iii{dLemented in this way, 
Autaiatic Tgst SayiOSDt imi. 

.Probably no single maintenance development has been so powerful, 
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and yet so ineffective, as has ME. A sophistica&d ATE routine can 

sanple hundreds of measurenients in a short time) and can race through 

a whole pre-launch sequence before a technician could Cfet a meter out 

of its case. Often the ATE system really can locate faults; seme 

programs even print out an English statement of just what needs to be 

done, and then "thank you" for doing the corrective work. Siych 

marvelous (^bilities lead to a trap, however, by^j^gesting that 

' since^ifiTE can do so mucKr it can do everytRing7 and skilled human 

troubleshooters will be unnecessary. This supposition is false. 

Recent history proves that ATE is extreKel^ vulnerable* to programming 

snarls, prime equipnent inaccessibility, the software difficulties of 

writing fault location routines, the incxanpatibility between 

autanatic and manual test procedures, and so forth. 

Many good analyses of ATE development issues have been published 

(e.g.., Myles, 1978; Van Kernel, 1978; SETE, 1965). The overselling of 

AOE capabilties still goes on, however. A recent example is the • 

Versatile Avionics* Shop Test (VAST) concept, Ttiis is an expensive 

and oonp^x general-purpose test -system for avionics. The system is 

an engineering tour de force which really can help the technician. 

But consider the following (|uote about the training of people who 

use VASt (Van Hemel,/ 1979): 

"Utte original concept, of VAST envisioned the use of an 
operator with minimal training,.,, This concept has been 
shown lacking because it anticipates a situation in. which 
the program will be perfect, The machine wil} always 
operate prc^rly, and docuroentaticm associated with the 
testing process will always be i^>-to-date and correct. 
Experience has shown that all of these factors seldom 
prevail in spite of the most stringent ef forts. , /.Ihe 
assumptions Originally nade concerning the VAST main- 
tenance technician also suffered from the same inac- 



62 



•/ 

curacies as with thfe curator.... this technician required 
rr»re training and experience to effectively troubleshoot 
the complex VAST systen," 

With enough money and effort, systems like VAST can be made to 
operate. But they seldom achieve theiV expectations, and they can be 
just another source of frustration to the f le^ people because they 
do not begin from a behavioral- requirement base. Indeed, an 
interesting resfearch project might be an attatipt to apply human 
factors and technician-behavior requirsnents to a complex ATE 
system, during the design biases. As far as we know, this has not 
yet been dc«§ on a systematic scale. ' 
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Training 

Readers of this r^rt will be familiar with the ordinary 
sequence, of technical schooiing-in the Services, and with the 
staggering numbers of courses, schools, and students. The DGD is 
probably the biggest schoolhouse in the world {Shriver^, 1975) . A 
recent Defense Sciences Board (Alluisi, 1976) concluded that 
/technical training in the U.S. military was usually effective, and 
one can accept that judgment. Froift a research standpoint, there are 
surprisingly few thorough evaluations of the training system. There 
are indeed many body counts ^ and schools often send out 
questionnaires to- the using commands, asking for data On the 
proficiencies and deficiencies of. school graduates. Such information 

I 

is presumably, used in modifying courses. ^ When genuine experimental 
evaluations are carried out, though, they are often done in a context 

m 

of a special training itan, such as an aircraft simulator. Since 
Realistic simulators are very expensive, it makes sense to find out 
whether they really save time in the aircraft. In this particular 
case, acceptable criterion measures can be defined, and the utility, 
and even the cost-effectiveness, of a training device can be 
estimated (Orlansky and String, 1976) . 

But with no criterion to guide most technicafi^ training courses, 
the usual curricular approach is to furnish general "theory" 
information, with the ejqjectation that this information will be the 
basis for on-the-job specialization later (Shriver, 1975)/^ It is 
hqped the technician can then figure out what to do, regardless of 
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conditions. Certainly it works this yiay someti-.^s, but often It does 

not. itie theory which is learned, and most likely quickly forgotten, 

may be so indirectly related to later activities; as to. be irrelevant, 

and there may be no direct training at all in the tasks that must be 

performed. The present^ training system is beginning to respond to 

such circumstances, although rather slowly, inaba { 1979) '^notes, for 

instance, that the Navy has started to teach the manuals: 

' "....the course essentially leads the stuaents 
through the manuals^but) the emphasis on using manuals 
in training has not really -helped much because the 
manuals are not usable. Once the student leaves the 
training environment, the manuals are put back on the 
^ V shelf again. 

rtie two main scientific difficulties with conventional training 
courses seem^to be (1) the lack of a criterion to assess the adequacy 
of the teaching, so that the. system is not self-correcting; and (2) 
the lack of a clear-cut behavioral ' requirement underlying the 
training plan. Instead, training plans begin with a set of 
assurrptions about background information that a person needs, to be 
able to learn on the job. . - 

«iese two difficulties conceivably could be alleviated, by 
research or by fiat. But there are administrative difficulties as 
well. To r^at an earlier point, the mUitary training systan 
is essentially a monopoly, with its own power and autonany.- it does 
not respond quickly to a new method or a new requiranent, because it 
does not have to. Also, a training unit roa^ not easUy accept inputs 
from, aayi the aiding people or the manual writers,\^in defining 
course content or practice schedules. The trainer may think that 
these other people want to eliminate training (Inaba, 1979) , and so 
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he may go ahead with his "general technical infonoation" approach. 
After all, that was way ha was taught, and the way he icnows 
best. Despite such oonsecvatisni, we should be slow to disparage 
conventional training.. It is often surprisingly difficult, for 
instance, to inprove upon conventional training with advanced 

« 

educational technology, as is shown in the next section^ 

C«rputers have been used a" practical school aid "for~nearIy~ 
fifteen years. Hie first successful ^plication of computer Aided 
Instruction (CM) was the Si^jpes-Atkinson project at Stanford, which 
g^^arithmetic and spelling lessons to young children, starting in 
1965. The Palo Alto classroom had 18 student stations, each one 

r . ' " 

equipped with its own terminal. In military training, there have 

it 

been about 30 studies of CAI-CMI (Computer Managed Instruction) 
effectiveness, and these have been collected and exhaustively 
ar^lyzed by Orlansky and string (1979) . Among their findings are the 
following: ^' . . 

a. Effectiveness has been measured only b/ testihg student 
'achievement at school, and jjQt by performance on the job. 

b. Student achievement in CAI-CMI courses is about the same 
as that observed under conventional instruction; 

c. CAI-CMI , Courses save about 30 percent (median value) of 
the time ' required to finish the same course given under 
conventional instruction; the time savings vary across 
courses and situations. 

d. Cost-effectivness of CAI-CMI has not been conclusively 
demonstrated, though some impressive "avoided cost" 
dollar savings can be claimed for the Navy OM systsn. 

'ttiese results, which are well documented in the Orlansky^String 

report, should temper any excessive enthusiasm for CAI-OH courses as 
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the kerto technical training efficiency. Of course, the course 
trials that have been reported may not represent the most advanced ' 
technology available. Truly radical CAi-au ijnprovanents may await 
letter psychological information^ ^ut the way students visualize and 
absorb oonplex physical and mathetimtical relations, or the way they 
learn a sequent;ial procedure. QeVer scheduling of lessons or "real- 
tijne monitoring of performance cannqk sul^titute for process 
infxjrmation "about the learner.* 

i - ' . 

\ I . 

1WO intriguing minor results came bit in/the Orlansky-String 
survey of military CAI-CMI. One of" these wal that CMI was about as 
effective as CM, in the saving of student training Ume. However. 
OM assistance oobts only five to seven cents \f^t student hour, 
whereas CAI costs a dollar or more per hour. On the basis of thts 
crude oonparison', CMI is some thirty times as cost-effective as 
CAI! Of course, there are difficulties with such a sinple-minded 
argimeht, but it i| 6till surprising that CAI should not do better 
than CMI. 

^ Ihe other finding was that instructors were ofteh unfavorable to 
both CM and 0!l. Many, people who wqgk in the CMSJuter-aiding field 
think they are helping the' instructors. For exan^jle, the claim is 
often made that by taking the drill-and-practice burden off the 
instructor's shoulders, the coraputer makes his job easier, and allows 
him to concentrate on more difficult coiK^^ual teaching. Why, then, 
are the instructors unfavorable, and are their negative atutudes due 
to sanething specific about the computer-aiding part of it, to 
imfmdliarity with the ^stm, or to perceived loss of course* 
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control? The matter deserves careful study. Regardless of how 
> coifuterized a course is/ there always must be human instructors 
around to load and run it, and to serve as consultants. * ^ 

Maintenance trainers and simulators are "poor sisters" in the 
simulator business. Kinkade (1979) points up the differences between 
the way training simulators are obtained for a new aircraft, and the 
w.^ they cojne about for maihtairters of "a new electronic device. In 
the aircraft case, simulatot dSrelcp^nt is initiated -early in 
procuranent, and is often budgeted and evaluated at the same time as the 
aircraft, ' The DO) aircraft oc»iinunity a^^has become accustomed to 
coirplex and expensive aircraft simulators, witk the average cost" 
running over a mill ion. dollars <Orlansky & String, 1977) . Ofvtxjurse, . 
such devices are often worth their high cost because of - the great 
expense and troulde jtf operating real aircraft. a result 6t these 
circumstances, the NTBC Directory of Naval Training Devices shows 
over 70% devoted to aircrew, 25% si^^jorting other operator training, 
and ^th less than 5% dedicated exclusively to maintenance 
technicians. Furthermore, of those devices that are designated as 

tedhnician trainers, • 

■ ■, . > 

. "•••.most are two-dimensional displays of system schematics, 
or 'cutaway* models of system equipments. Very few provide . * 
for 'hdinds-on* practice of technician tasks and these almost 
exclusively involve simulation of actjial system equipments." , 

^^tudies have indicated that iteintenanpe trafcr simulators can 

result in high savings, conpared- to using the priipe equi(iaent . ^ 

J (Daniels & Cronin, 1975; Miller & Rockway, 1975) . Also, use of reaX 

^rime equipment Aas obvious limitations. Th^ hardware will require 
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"peaking up" between training sessions; it is difficult to observe 
and score performances; the training model which is available may not 
reflect the realities of field installation, and so forth. Several 
projects have designed complex maintenance simulator trainers, and 
COTducted sane preliminary evaluations. Among these is the Fault 
Identification Simulator (FIS) used in an experimental course tOr 
automatic boiler control technicians. The device can show, in 
graphic fashion, the symptoms of 23 different fault conditions. In 
fact, all possible fault conditions resulting from a single component 
failure in the actual ^stem can be simulated in the FIS (Kinkade, 
1979) . This trainer also advises the students of incorrect repair 
decisions about a non-faulty element. Since the simulator is driven 
by a small computer, it can be programmed to reflect changes in the 
.prime equipnent. When tried out as part of an experimental course, 
the class finished training in two or three weeks, as ooniared to a 
regular course duration of five weeks (the experijnental class covered 
the same material as the usual course, and even had more 
troubleshooting practice than in the old system). • 

.Shepherd et al., (1977) sinailated the oontifol panel layout of a 
tihemical plant, and taught trainees how to recognize faults in 
three different ways: (i) a theory approach* based on oonyiBntional 
instruction; (2) a rules model, *|herein siijects were given rules that 
would help than infer faUures from "the panel arr^y and (3) a "no 
story" or control group received no theoretical training. The 
rules" groupe was better able to diagnose unfamUiar faults (80% 
acajracy, vs. 65% ana 49t for theory and control). One conclusion is 
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worth quoting here/ for its reiteration of a previous point: 

"....there is little to be gained by teaching •theory* 
alone. Although trainees in the 'theory* groifl? initially 
performed well on unfamiliar faults, their score, by the 
final test, had deteriorated to a level not significantly 
different from the 'no stdry* group." 

The AIDE and lOGMOD systems, and other advanced aiding devices 

♦ 

with large memory, also can be used as simulator trainers, in fact, 
pure (exclusive) training and pure perf ontance aiding \ze rare 
occurrences. Mogt training e^riences involve some degree of job 
support, and mpst si^jported performances rely on some degreee of ' 
previous training. 

Another training concept is the Generalized Maintenance Trainer 
Simulator (OfTS), (Rigney et al., 1978). This system- 
was designed to give students intensive troubleshooting practice in a 
simulated Woids-on training environment. Its "simulational \ 
bandwidth" is confined to conditions and operations necessary for ' 
practicing fault localization, either f rcjin front panel information or 
from data at test-points behind the front panel. QfES adapts readily 
to simulate different kinds of equipment, by accepting data bases 
that describe functional and structural architecture. As a trainer, 
the GffTS functions autonatically. Once a student signs on, he gets 
* instructions, his progress i^ tracked, and a detailed performance 
record is kept. In a trial with 16 Navy technicians, practice with 
the QfCS reduced average times to locate UHF malfunctions from 20 
minutes to less than 10 minutes. Most subjects rated it favorably as. 
a learning aid (Rigi«y, et al,, ld7§), 

the SCPHIE training system is one of the most "reactive* of the present 

.. : ■ ' ,1 : 70 ' ■ ■ 
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simulator configurations. Subjects caif change coirponent values in the 
circuit under study, can propose hypotheses to an "articulate expert", 
and can play a "troubleshooting game" by predicting the effects- of . 
inserted faults. Students respond very positively to this lejiming 
environment {Brown, et al., '1975). 'The simulated tutor usually provides 
explanations of student mistakes. Some tutors even proceed in a 
Socratic fashion, by challenging the student so that he corrects hiinself 
(Stevens & Collins, 1977) . 

■niese several recent experiences with complex maintenance trainers . 
are most encouraging. ARaarently, special maintenance simulators arfe 
able to teach technicians efficiently about malfunctions. We believe 
they work as well as they*'do because of the clear explication of 
syitptont-fault patterns, and because of the intelligibility appeal 
.mentioned by shriver (1975). Thus, they meet Duncan's suggestion 
that, for, teaching trouble diagnosis, training simulators should 
preserve the panel layout and also the "logical layout" of the 
indicators (Duncan et al,, .1975b) , 

•ttie leading research on QJT is probably being done at the Rand 
Corporation, where Carpenter-Huffman (1979) and others have been 
^rveying field units, instructors, )uid trainees. Many deficiencies 
in present QJT pracftices are rather to be eaqaected: The training is 
secondary to operational needs of the inoinent, an adequate instructor 
nay not be present, there is a lack of sqpplies for training purposes, 
Uiere is also concern over the instructorrtype skills of the trainers 
(Gacpepter-Huffnan, 1979)?, * 
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"The most orawnon single oonplaint about cxrr on the mc 
quest iionnai re was that those assi^ed as trainer were 
inadequate teachers," 

underlying these rather^inevitable deficiencies are sane 
adninistrative causes. Military connanders, 

"....are not iirroediately rewarded for the quality of their 
maintenance activities, let%lone their maintenance GOT 
programs. Instead, they and those they conmand are 
penalized if documentation of progress in QJT does not 
conform to the schedule set for it..., pressures to come in 
as Categorty 1 on the Operational Readiness report often 
destroy the validity and' us^ulness of vehicle preventive 
maintenance and operational readiries records.... On the 
other hand, since progress in COT is tied to the promotion 
system, supervisors are under pressure to certify a person's 
- conqpetence whether or not the certification is warranted." 

t * 

The Rand approach to OJT involves a more formal, explicit, and 
reinforcable conception of COT within ojperating units. Several 
successful applications appear to follow this approach, including the 
Navy FBfiMP and the TOT (Task-Oriented Training) projects in the Air 
' Force Tactical Air Command, Though this area is perhaps not as 
J exciting as some of the new aiding concepts, it is receiving some 
attention, and the main requirements have been identified. 
Administrative decisions will be the key factors, since the military 
can have effective QJT if it really wants it. 



SECTION IV. JOB DESIGN FOR THE ^TRDUBLESHOOTER. * 



Job design is the organization of a job to satisfy the 
tecfanical'organization requirements of the work to be accomplished 
and the human requirements of the person performing the work" (Davis 
& KantJ^r, 1955) . Hbe second part of this definition is the key one. 
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as it directs attention to the fit between the work and the needs of 
the worker. Many proposals have been offered, for inproving the fit. 



\ 
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For example, Davis listed twentj^three techniques v^ich might 
simultaneously increase worker satisfaction and lead to more 
effective output (Davis 1957) . Man^ of his proposals have to do with 
increasing worker autonoty, discretion, achievement, growth, and the 



meaningfulness and variety, of tasks. It is surprising how modern 
these ideas sound after more than twenty years. 



Despite the absence of definitive survey data, it may be 



worthwhile, to speculate about the present" job-design status of the 
maintenance technician's work. For this purpose, we consider the 
three job characteristics that Hackman and Lawler (1971) identify as 
contributing to worker need satisfaction and to organizational 
goals. Their first attribute is that the worker must feel personally 
responsible for his work. On this dimension, the high-technology * 
troubleshooter should score fairly high.' Take the conputer 
maintenance man. -^e often works alone, and he may not turn to other 
people for help, becaus| th^ cannot help. At the same time, however, 
he certainly must'Jaww that he. is priitiarily a skilled user of 
materials which were originated and validated by others, and that he 
is not ordinarily free to vjry the procedures very much. Computer 
techrticians, for exanple, are not allowed to change maintenance 



software, though we have heard of sane doing so. On Gulowsen's (1979) 
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is perceived as meaningful and worthwhile to ^e individual. 
According to Turner and Lawrence (1965) ^ meaningful job processes 
have clear beginning and ending of the work, they utilize skills and 
abilities which the worker personally values, ai^ they require 
considerable variety. On these grounds, military troubleshooters 
certainly should perceive their work as meaningful. They often have 
cofimand pressure on them, and this accentuates the value of the work 
being performed. . 

Feedback is a third job-design criterion. A person vrtio is having 
higher-order needs satisfied wants to' know how he or she is doing. 
Superficially, the troubleshooter gets almost perfect objective 
feedback, because the failed machine, either is restored or it is not, 
and the technician usually knows which state prevails (but not 
always, as we indicated earlier) . This iintiediate task feedback, of 
course, is not necessarily identical with- the performer's own 
perception of his effectiveness, sinc^ he may be evaluating himself 
against some other standard. 

On these three criteria, then, the technician should be rather 
highly motivated in his work, and from superficial indications he may 
indeed be quite satisfied, compared to other military people, 
CcHiparative personnel statistics are difficult to find, but one large 
conputer company r^rted that turnover and absence rates for 
conmercial maintenance men are very low, lower than for design 
engineers m. progranmers. Ihis is especially so for medium-size 
cities in America. A few on-site interviews with roving oamercial 
confxiter technicians did reveal a couple of coninon sources of - 



dissatisfaction. One of thege was the technician's non-professional 
status, which was related to what pne person does. Though the work is 
technical aqd demanding, experience in restoring conpjters results in 
a person's learning more and more about one particular company's 
coinputers, manuals, and fault-isolation procedures. But it does not 
result in general engineering knowledge. Thus, many technicians 
perceive that they are regarded as "less than engineers," and that 
they can never attain engineer status, regardless of how expert they , 
become at their work. The military services may be massing an 
o^Jortunity to satisfy these growth needs of their technicians. A 
similar growth-need situation was observ^ in the U.S. aerospace 
industry. .Engineers in that industry perceived that their real worth 
depended in their state-of-the-art design capabilities. As a result, 
when design work was to be done, they tended to design all-new itans, 
and to reject existing, and off-the-shelf hardware. They did this to 
iirprove themselves and to keep up-to-date, but it was very costly to 
the company. A solution was for the firm to consider both the 
engineer's perceived to sharpen intellectual skills, and the 
denands of the rcHitine design work *^ich had to be done. Managonent 
guaranteed that an individual would have career-advancing assignments 
often, though at any one time he mighfe be doing a routine job (Davis 
& Taylor, 1979) . A similar job design strategy could be used with 
military maintenance pec^le, 

A more subUe negative factor which was f requenUy noted in the 

« 

in&istirial setting was the psychological distance between the 
n^ntenance technicians, and other pec^de in the cdnqputer centers. 

* ^ . ♦ 

^ V 75. . . ■ 
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The TGving maintenance person may have few close relationships with 

other workers, as he is always moving from one place to another, and 

the other technicians back at maintenance headquarters are also 

usually on the move. Some respondents even confided that the work 

leads to a general sense of isolation and non-sociability. 

As a final job-design variable, we mention the "flow" experience, 

which can be a profound dynamic state {Csikszentmihalyi, 1975) : 

/ "....the holistic sensation that people feel when they 
act with total involvement,.,. (the person) experiences 
it as a unified flowing from one moment to the next, 
in which he is in control of his actions, and in which 
there is little distinction between self and - 
environment, between stimulus and response, or between 
past, present/»^4nd future.,,. People seek flow 
primarily fm itself, not for the incidental extrinsic 
rewards tilat may, occur from it." 

Could a tech(iician*s search task be so arranged that this 

autotelic-fiow state could occur with some regularity? If so, the 

irbtivationSr^dvantages should be enormous.. Prom all accounts, flow 

activities are avidly sought the people who have experienced 

them. A beginning could be made by analyzing the flow potentflal in 

certain maintenance activities. 

X 

/ 

ACTION V. REPRISE AND REOOMMENtATIONS 

/ ■ ■ ■ ■ 

Hie military troubleshooter is a fairly good- searcher and 
reasoner. What he needs most is a clear model of the functional 
relations among syn^oros he observe and malfunctions in the prime 
eguipnent, perhaps acoonpanied by a sequencing model for ordering * 
test actions. Often it is inf>racticable for him to learn or reroefnber 

•| ■ - ■ 
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enough theory to generate this model for himself, and so the model 
will have to be furnished tc| him. The best procedure for doing this 
so far is to have the prime jequipinent analyzed in such a way that 
all, or nearly all, roalfunctjioning states of the equipment can^^e^^ 
correlated with observable symptems. The states and syn^xtoros are 
then stored in some way that can be systematically searched via a 
predetermined sequence, the main job of the technician then becomes 
the highly skilled task of searching via complex instructions and 
diagnostic aids. 

"New lode" formats for aiding and training these skills have been 
tried out in the laboratory and in the field, and tSiey are clearly 
more effective than conventional training plus QJT. The critical 
requirement is the accuracy and usefulness of the information 
presented, rather than the particular display medium. Present 
carputer technology can store and distribute, in either hand-held or 
larger portable units, all the information that is ordinarily 
included in book-size manuals, along with test sequencing and 
evaluating data. This technology can, then, alleviate maintenance 
pr(|>blenis' on most oonplfex syfeteni^, if it is designed and aE?>lied with 
the joehavioral requirements in mind. 

W military should not be satisfied with procedurally aided 
troubleshooting, regardless of how effective a good aiding system 
may be. Many maintenance specialists will have to possess real 
insight into the cxnplicated devices of modem warfare, in order to 
attack the unforeseeable problems. Tfis provision of these more 
advanced peqple will be a continuing challenge to the research and 
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training oanmunities, but there is reason for optiroiam, tie know much 
more about the interaction of men and machines than we did in the 
vacuum-tube era. 

* 

Since effective technology (psychological; aiding and 

maintainability) is indeed available, and we still have serious 

i 

probl^, then there must be administrative or other reasons why the 
knowledge has not been more fully utilized (Bond, 1970) . We believe 
that it is not cost that prevents application. Some "new look*' 
aiding and training materials do require significant proration and 
ddxigging effort, ^but these are one-tijne expenses, and they are 
insignificant compared to present wasted manpower and downtime 
losses. The administrative key is thi^: To develop and to 
in^Jlement effective aiding and training systems, many separate 
efforts have to be coordinated and directed through a central 
"corrective maintenance control" concept. This has been done in the 
coiroercial con^JUter world and in other domains, but it cannot be done 
easily in the military, because no single command or agency is really 
responsible for it and can insist that a valid maintenance o0ncept be 
realized. The procuring, designing, and aiding and training (lave to 
be planned together and acoon^lished together, and this infrequently 
happens in the present military system. 

In a report addressed to the research oorammity, most of our 
"reooiiinendations" come in the form of projects that might tfe 
undertaken, A few of these have to do with basic research into human 
appreciation of complex devices. Others attend to further the 
production of arrangonents which will facilitate performance, and 
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thus bypass the psycholpgical problems. No doubt some of thes4 ideas 
are already being pursued. The projects are not listed in order of 
inportance. For wro^iat^ pg^gt|<?aJ. importance to ndlitary 
effectiveness, numbers 4, 5, 10, 17, and 19 wodld be near- the top of 
the order. As far as psychologtoil significance numbers 7, i;^, 13, 
14, and 16 are most interesting. 

I ■ 

i 

Research Recommendations: Possible Projects in Troubleshooting 

1. Investigate the use of pseudo-verbal software routines (e.g., 
Findler's Puzzle-Solver, Wang's Theorem+Prover) for fault^^location 
or aiding, to determine if the pseudo-Er|glish format is suitable 

for practical search tasks. 1 ^ . » 

2. Explicate the basic relations between suih methodologically 
similar concepts^such. as Cc^^Ott optimal testing, Fano coding, 
Huffman coding, switchl^Jt^^Y diagnosis of logical networks, 
and so forth. (Some of the-^idfentities among these methods have 
already been proven, but ndsody has piit them all together for the 

* P^chological research ocranunity.) 

3. Explore the extent ^ which small handheld aids, such as 
troubleshooting sequence cards or wheels, are now applied by maitary 
technicians. 

4. Find out how to best utilizie hand-held ccaiputer technology for 
troubleshooting. This would include consideration of the sizes 
of proceduralized logic trees that could be stored, the. best ways 
to utilize display and recording techniques, acceptability of the 

• .^ice to field peqple, and develc^inent'ooists. 
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5. compare the effectiveness of "new look" proceduralized 
naintena^ice nianuals, as presented via a LOGNCD or AIDE 
ccraputerized configuration, with the same information given in a 
series of booklet^. 

6. Study the feasibility of a strwig "Maintenance AH>lications 
Group" in the laboratories for each service, and see what rewards 
coul,d be given for high-quality applications research, within 
civil-service constraints. 

7.. Determine suitable roetJjfi^ for measuring cognitive style in 

technical work, and study the possibilities for exploiting individual 
differoioBs on this variable, in the maintenance domain, 

8. Review a suitable sample of Automatic Itest Equipment (ATE) 
routines, and delineate the usefulness, and the limitations, of 
these routines to ordinary military technicians. ^ 

9. According to Orlansky and String (1979), there are about 30 
stiKiies of CAI and CHI effectiveness. Determine* how closely the 
CAT and CMI technology there reviewed is "up to the best present 
state-of-the-art". 

10. Discover the reasons for unfavorable ratingSjOf CAI-Ott programs 
by the instructors. 

11. Study the process by which good ideas are "stopped^ in the 
maintenance world. Suppos^^^emebody proves that a sequenticQ or " 
logic-tree troubleshooting scheme is better on a militar^ 

radar or computer (in fact. Potter's group did this) . Determine 
why the technology was not e;^loited more fully. 

12. Determine the extent to which electronics or electromechanical 

•I 
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theory is remembered, after schooling. A project could test 
at several intervals, say, 3, 6, and 24 months after schooling 

i 

is finished. 

13. Collect all the means for helping students to visualize circuit 
and electro-niechanical relations, and evaluate ea^rimentally the 
most promising of these methods. There are many ingenious 
prc^sals from mathematics and physics teachers, and from sunmer 
science workshops. Sane of these are worth experimental 
inveistigation, and could be correlated to the literature in 

e 

cognitive psychology. 

14. Investigate the utility of •'mnemonics for electronics," in 

• memorizing difficult material such as circuit algebraic laws. 

15. Find out if the time it, takes a technician to do a basic digital- 
logic operation (via Trabasso or Posner timing techniques) is a 
reliable information-processing indicant, and if so, whether it 
is correlated with troubleshooting performance. 

16. Effljlorethe "flow" and "play" aspects of troubleshooting 
;rformance, according to the Csikszentmihalyi paradigm, and see 

if aids can be constructed that could enhance the likelihood of 
these flow effects in troubleshooting. • 

17. Do an extensive ooriparison of military troubleshooting with that 
performed by ooninercial coraputer servic^ oonpanies, with ^cial 
attention to "what the inilitaryf can /learn" from the cannercial 
experience. / 

18. See if ^)ecial training in branch matrix reasoning, can transfer 
positively to logical (troubleshooting) search iJi realistic s-M 
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tables. ' , - 
J.9. Determine the effectiveness curves- of intensive training (more. ' 

than 100 troubleshooting problems, fierhaps) in using a oonqplex 

diagnostic aid on subsequent troubleshooting performance, 
20. Originate "story line" interpretations of coii5>le>| physical events, 
I ' say in AC circuit theory, and evaluate these for their enhancement 

of learning, , 

• * 

■ . 

Administrative Reconmendations 

1. Conduct an unbiased survey of present prime-equipnent 

» 

t capabilities. To prevent the usual "softening" or blurring of 
lack of capability, this survey probably should be done by an 
outside engineering group. Sampling techniques would permit 
strong inferences about true capabilities. 

2. Discover and remove non-maintainable -equipment items. This 

\; action would be an^obvioua sequel to the capability survey. It 
should relieve operating cranraands from the burden of carrying non- 
us^le major, equipnents. ' ^ 
3. . doncentrate first-ailistatient technician activities on the es^rt 
use of fully-proc^uralized troublesh^ting aids, and drastically 
reduce or eliminate the piresent -theory-oriented courses* -ildiey 
(1978) shows hoW 'thi? personnel decision alone wdW-d »ve . mill ions of 
dcCtld^rs in "effective "technician" work time. Hard data supports the 
idea; in addition to the small evai^uation studies, the services 
should undertalce massive trials of procedQralized aids, and this 
probably should be done outside the present training coRrounity. 



Develop a flying squad" troubleshooSig concept, ma^e \sp of 
• truly expert technicians. Unlike th^ first-enlistment people, 
these second-term people w6uld receive extensive theoretical * 
trairfing. The squad would handle the 'tough field problans, that 
>is, the reniaining trpubles that cannot be readily located by 
pr^eduralizedtaids.' We do not' refer here to simple seniority, 
such as a cadre of first class ETTs with conventional training; 
we have many of those now who cannot solve maintenance problems. 
As in the fully pro9edur aliped concept for first- termers, the 
squad concept . would -be oriented to effective performance, ii^ much 
the same way as osmiercial conputer trouble squads are managed. 
The prospects of qualifying for thi's kind of work should be 
motivating to the best career technicians, and could lead to 

professional-level traifining in the second and later enlistments. 

— - ■— - 

Plan for high-level military administrative. "fixers" who support 
a maintenance program^ and continuously guide it through the 
financial and administrative difficulties faced by every change. 
A finer might be an adniral or general. He would develop a % 
maintenance constituency for supporting the use of spec^l 
troubleshooting aids, for exanple; He would also intervene with 
agency officials on ^cial problems that arise, such as finding 
rocjney to debug a training concept or hand-held ccapiter aid, or 
pursuing imp^nentation plans in difficult environments. He 
tuould be able' to move across departsoents ai^ agencies, and would 
be able to influence and control activities so that the present 
technology" is utilized. 
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